ray.data.llm.PrepareMultimodalStageConfig#

class ray.data.llm.PrepareMultimodalStageConfig(*, enabled: bool = True, batch_size: int | None = None, concurrency: int | Tuple[int, int] | None = None, runtime_env: Dict[str, Any] | None = None, num_cpus: float | None = None, memory: float | None = None, model_config_kwargs: Dict[str, Any] | None = None, chat_template_content_format: Literal['string', 'openai'] | None = 'string', apply_sys_msg_formatting: bool | None = False)[source]#

The configuration for the prepare multimodal stage.

Parameters:

  • enabled – Whether this stage is enabled. Defaults to True.

  • model_config_kwargs – Optional kwargs to pass to the model config. See available model config kwargs at https://docs.vllm.ai/en/latest/api/vllm/config/#vllm.config.ModelConfig.

  • chat_template_content_format – The content format to use for the chat template. This is used to format the chat template content according to a specific model. Choices are “string” or “openai”. Defaults to “string”.

  • apply_sys_msg_formatting – Whether to apply formatting system messages. Defaults to False.

  • batch_size – Rows per batch. If not specified, will use the processor-level batch_size.

  • concurrency – Actor pool size or range for this stage. If not specified, will use the processor-level concurrency. If concurrency is a tuple (m, n), Ray creates an autoscaling actor pool that scales between m and n workers (1 <= m <= n). If concurrency is an int n, CPU stages use an autoscaling pool from (1, n).

  • runtime_env – Optional runtime environment for this stage. If not specified, will use the processor-level runtime_env. See this doc for more details.

  • num_cpus – Number of CPUs to reserve for each map worker in this stage.

  • memory – Heap memory in bytes to reserve for each map worker in this stage.

PublicAPI (beta): This API is in beta and may change before becoming stable.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'protected_namespaces': ()}#

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].