ray.data.llm.PrepareMultimodalStageConfig#

class ray.data.llm.PrepareMultimodalStageConfig(*, enabled: bool = True, batch_size: int | None = None, concurrency: int | Tuple[int, int] | None = None, runtime_env: Dict[str, Any] | None = None, num_cpus: float | None = None, memory: float | None = None, model_config_kwargs: Dict[str, Any] | None = None, chat_template_content_format: Literal['string', 'openai'] | None = 'string', apply_sys_msg_formatting: bool | None = False)[source]#

The configuration for the prepare multimodal stage.

Parameters:

  • enabled – Whether this stage is enabled. Defaults to True.

  • model_config_kwargs – Optional kwargs to pass to the model config. See available model config kwargs at https://docs.vllm.ai/en/latest/api/vllm/config/#vllm.config.ModelConfig.

  • chat_template_content_format – The content format to use for the chat template. This is used to format the chat template content according to a specific model. Choices are “string” or “openai”. Defaults to “string”.

  • apply_sys_msg_formatting – Whether to apply formatting system messages. Defaults to False.

  • batch_size – Rows per batch. If not specified, will use the processor-level batch_size.

  • concurrency – Actor pool size or range for this stage. If not specified, will use the processor-level concurrency. If concurrency is a tuple (m, n), Ray creates an autoscaling actor pool that scales between m and n workers (1 <= m <= n). If concurrency is an int n, CPU stages use an autoscaling pool from (1, n).

  • runtime_env – Optional runtime environment for this stage. If not specified, will use the processor-level runtime_env. See this doc for more details.

  • num_cpus – Number of CPUs to reserve for each map worker in this stage.

  • memory – Heap memory in bytes to reserve for each map worker in this stage.

PublicAPI (alpha): This API is in alpha and may change before becoming stable.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'protected_namespaces': ()}#

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'apply_sys_msg_formatting': FieldInfo(annotation=Union[bool, NoneType], required=False, default=False, description='Whether to apply formatting system messages.'), 'batch_size': FieldInfo(annotation=Union[int, NoneType], required=False, description='Rows per batch.'), 'chat_template_content_format': FieldInfo(annotation=Union[Literal['string', 'openai'], NoneType], required=False, default='string', description='The content format to use for the chat template. This is used to format the chat template content according to a specific model.'), 'concurrency': FieldInfo(annotation=Union[int, Tuple[int, int], NoneType], required=False, description='Actor pool size or range for this stage.'), 'enabled': FieldInfo(annotation=bool, required=False, default=True, description='Whether this stage is enabled.'), 'memory': FieldInfo(annotation=Union[float, NoneType], required=False, description='Heap memory in bytes to reserve for each map worker in this stage.'), 'model_config_kwargs': FieldInfo(annotation=Union[Dict[str, Any], NoneType], required=False, description='Optional kwargs to pass to the model config. See available model config kwargs at https://docs.vllm.ai/en/latest/api/vllm/config/#vllm.config.ModelConfig'), 'num_cpus': FieldInfo(annotation=Union[float, NoneType], required=False, description='Number of CPUs to reserve for each map worker in this stage.'), 'runtime_env': FieldInfo(annotation=Union[Dict[str, Any], NoneType], required=False, description='Optional runtime env for this stage.')}#

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.