ray.data.extensions.tensor_extension.ArrowVariableShapedTensorType

class ray.data.extensions.tensor_extension.ArrowVariableShapedTensorType(dtype: pyarrow.lib.DataType, ndim: int)

Arrow ExtensionType for an array of heterogeneous-shaped, homogeneous-typed tensors.

This is the Arrow side of TensorDtype for tensor elements with different shapes. Note that this extension only supports non-ragged tensor elements; i.e., when considering each tensor element in isolation, they must have a well-defined, non-ragged shape.

See Arrow extension type docs: https://arrow.apache.org/docs/python/extending_types.html#defining-extension-types-user-defined-types

PublicAPI (alpha): This API is in alpha and may change before becoming stable.

__init__(dtype: pyarrow.lib.DataType, ndim: int)

Construct the Arrow extension type for array of heterogeneous-shaped tensors.

Parameters

• dtype – pyarrow dtype of tensor elements.

• ndim – The number of dimensions in the tensor elements.

Methods

__init__(dtype, ndim)	Construct the Arrow extension type for array of heterogeneous-shaped tensors.
equals(self, other, *[, check_metadata])	Return true if type is equivalent to passed value.
field(self, i)
to_pandas_dtype()	Convert Arrow extension type to corresponding Pandas dtype.
wrap_array(self, storage)	Wrap the given storage array as an extension array.

Attributes

bit_width
extension_name	The extension type name.
id
ndim	Return the number of dimensions in the tensor elements.
num_buffers	Number of data buffers required to construct Array type excluding children.
num_fields	The number of child fields.
scalar_type	Returns the type of the underlying tensor elements.
storage_type	The underlying storage type.