ray.data.extensions.tensor_extension.ArrowTensorArray#

class ray.data.extensions.tensor_extension.ArrowTensorArray[source]#

An array of fixed-shape, homogeneous-typed tensors.

This is the Arrow side of TensorArray.

See Arrow docs for customizing extension arrays: https://arrow.apache.org/docs/python/extending_types.html#custom-extension-array-class

PublicAPI (beta): This API is in beta and may change before becoming stable.

__init__(*args, **kwargs)#

Methods

__init__(*args, **kwargs)

buffers(self)

Return a list of Buffer objects pointing to this array's physical storage.

cast(self[, target_type, safe, options])

Cast array values to another data type

dictionary_encode(self[, null_encoding])

Compute dictionary-encoded representation of array.

diff(self, Array other)

Compare contents of this array against another one.

drop_null(self)

Remove missing values from an array.

equals(self, Array other)

fill_null(self, fill_value)

See pyarrow.compute.fill_null() for usage.

filter(self, Array mask, *[, ...])

Select values from an array.

format(self, **kwargs)

from_buffers(DataType type, length, buffers)

Construct an Array from a sequence of buffers.

from_numpy(arr)

Convert an ndarray or an iterable of ndarrays to an array of homogeneous-typed tensors.

from_pandas(obj[, mask, type])

Convert pandas.Series to an Arrow Array.

from_storage(BaseExtensionType typ, ...)

Construct ExtensionArray from type and storage array.

get_total_buffer_size(self)

The sum of bytes in each buffer referenced by the array.

index(self, value[, start, end, memory_pool])

Find the first index of a value.

is_null(self, *[, nan_is_null])

Return BooleanArray indicating the null values.

is_valid(self)

Return BooleanArray indicating the non-null values.

slice(self[, offset, length])

Compute zero-copy slice of this array.

sort(self[, order])

Sort the Array

sum(self, **kwargs)

Sum the values in a numerical array.

take(self, indices)

Select values from an array.

to_numpy([zero_copy_only])

Convert the entire array of tensors into a single ndarray.

to_pandas(self[, memory_pool, categories, ...])

Convert to a pandas-compatible NumPy array or DataFrame, as appropriate

to_pylist(self)

Convert to a list of native Python objects.

to_string(self, *, int indent=2, ...)

Render a "pretty-printed" string representation of the Array.

to_variable_shaped_tensor_array()

Convert this tensor array to a variable-shaped tensor array.

tolist(self)

Alias of to_pylist for compatibility with NumPy.

unique(self)

Compute distinct elements in array.

validate(self, *[, full])

Perform validation checks.

value_counts(self)

Compute counts of unique elements in array.

view(self, target_type)

Return zero-copy "view" of array as another data type.

Attributes

nbytes

Total number of bytes consumed by the elements of the array.

null_count

offset

A relative position into another array's data.

storage

type