ray.serve.air_integrations.PredictorWrapper#

class ray.serve.air_integrations.PredictorWrapper(predictor_cls: Union[str, Type[Predictor]], checkpoint: Union[Checkpoint, str], http_adapter: Union[str, Callable[[Any], Any]] = 'ray.serve.http_adapters.json_to_ndarray', batching_params: Optional[Union[Dict[str, int], bool]] = None, predict_kwargs: Optional[Dict[str, Any]] = None, **predictor_from_checkpoint_kwargs)[source]#

Bases: ray.serve.air_integrations.SimpleSchemaIngress

Serve any Ray AIR predictor from an AIR checkpoint.

Parameters

  • predictor_cls – The class or path for predictor class. The type must be a subclass of ray.train.predictor.Predictor.

  • checkpoint –

    The checkpoint object or a uri to load checkpoint from

    • The checkpoint object must be an instance of ray.air.checkpoint.Checkpoint.

    • The uri string will be called to construct a checkpoint object using Checkpoint.from_uri("uri_to_load_from").

  • http_adapter – The FastAPI input conversion function. By default, Serve will use the NdArray schema and convert to numpy array. You can pass in any FastAPI dependency resolver that returns an array. When you pass in a string, Serve will import it. Please refer to Serve HTTP adatpers documentation to learn more.

  • batching_params – override the default parameters to ray.serve.batch(). Pass False to disable batching.

  • predict_kwargs – optional keyword arguments passed to the Predictor.predict method upon each call.

  • **predictor_from_checkpoint_kwargs – Additional keyword arguments passed to the Predictor.from_checkpoint() call.

PublicAPI (alpha): This API is in alpha and may change before becoming stable.

async predict(inp)[source]#

Perform inference directly without HTTP.

reconfigure(config)[source]#

Reconfigure model from config checkpoint