ray.data.llm.DetokenizeStageConfig#

class ray.data.llm.DetokenizeStageConfig(*, enabled: bool = True, batch_size: int | None = None, concurrency: int | Tuple[int, int] | None = None, runtime_env: Dict[str, Any] | None = None, num_cpus: float | None = None, memory: float | None = None, model_source: str | None = None)[source]#

The configuration for the detokenize stage.

Parameters:

  • enabled – Whether this stage is enabled. Defaults to True.

  • model_source – Model source/identifier for this stage. If not specified, will use the processor-level model_source.

  • batch_size – Rows per batch. If not specified, will use the processor-level batch_size.

  • concurrency – Actor pool size or range for this stage. If not specified, will use the processor-level concurrency. If concurrency is a tuple (m, n), Ray creates an autoscaling actor pool that scales between m and n workers (1 <= m <= n). If concurrency is an int n, CPU stages use an autoscaling pool from (1, n).

  • runtime_env – Optional runtime environment for this stage. If not specified, will use the processor-level runtime_env. See this doc for more details.

  • num_cpus – Number of CPUs to reserve for each map worker in this stage.

  • memory – Heap memory in bytes to reserve for each map worker in this stage.

PublicAPI (alpha): This API is in alpha and may change before becoming stable.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'protected_namespaces': ()}#

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'batch_size': FieldInfo(annotation=Union[int, NoneType], required=False, description='Rows per batch.'), 'concurrency': FieldInfo(annotation=Union[int, Tuple[int, int], NoneType], required=False, description='Actor pool size or range for this stage.'), 'enabled': FieldInfo(annotation=bool, required=False, default=True, description='Whether this stage is enabled.'), 'memory': FieldInfo(annotation=Union[float, NoneType], required=False, description='Heap memory in bytes to reserve for each map worker in this stage.'), 'model_source': FieldInfo(annotation=Union[str, NoneType], required=False, description='Model source/identifier for this stage.'), 'num_cpus': FieldInfo(annotation=Union[float, NoneType], required=False, description='Number of CPUs to reserve for each map worker in this stage.'), 'runtime_env': FieldInfo(annotation=Union[Dict[str, Any], NoneType], required=False, description='Optional runtime env for this stage.')}#

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.