ray.data.extensions.tensor_extension.ArrowTensorType#

class ray.data.extensions.tensor_extension.ArrowTensorType(shape: Tuple[int, ...], dtype: pyarrow.lib.DataType)[source]#

Arrow ExtensionType for an array of fixed-shaped, homogeneous-typed tensors.

This is the Arrow side of TensorDtype.

See Arrow extension type docs: https://arrow.apache.org/docs/python/extending_types.html#defining-extension-types-user-defined-types

PublicAPI (beta): This API is in beta and may change before becoming stable.

__init__(shape: Tuple[int, ...], dtype: pyarrow.lib.DataType)[source]#

Construct the Arrow extension type for array of fixed-shaped tensors.

Parameters
  • shape – Shape of contained tensors.

  • dtype – pyarrow dtype of tensor elements.

Methods

__init__(shape, dtype)

Construct the Arrow extension type for array of fixed-shaped tensors.

equals(self, other, *[, check_metadata])

Return true if type is equivalent to passed value.

field(self, i)

to_pandas_dtype()

Convert Arrow extension type to corresponding Pandas dtype.

wrap_array(self, storage)

Wrap the given storage array as an extension array.

Attributes

bit_width

extension_name

The extension type name.

id

num_buffers

Number of data buffers required to construct Array type excluding children.

num_fields

The number of child fields.

scalar_type

Returns the type of the underlying tensor elements.

shape

Shape of contained tensors.

storage_type

The underlying storage type.