Skip to main content
Ctrl+K

Site Navigation

  • Get Started

  • Use Cases

  • Example Gallery

  • Library

    • Ray CoreScale general Python applications

    • Ray DataScale data ingest and preprocessing

    • Ray TrainScale machine learning training

    • Ray TuneScale hyperparameter tuning

    • Ray ServeScale model serving

    • Ray RLlibScale reinforcement learning

  • Docs

  • Resources

    • Discussion ForumGet your Ray questions answered

    • TrainingHands-on learning

    • BlogUpdates, best practices, user-stories

    • EventsWebinars, meetups, office hours

    • Success StoriesReal-world workload examples

    • EcosystemLibraries integrated with Ray

    • CommunityConnect with us

Try Ray on Anyscale

Site Navigation

  • Get Started

  • Use Cases

  • Example Gallery

  • Library

    • Ray CoreScale general Python applications

    • Ray DataScale data ingest and preprocessing

    • Ray TrainScale machine learning training

    • Ray TuneScale hyperparameter tuning

    • Ray ServeScale model serving

    • Ray RLlibScale reinforcement learning

  • Docs

  • Resources

    • Discussion ForumGet your Ray questions answered

    • TrainingHands-on learning

    • BlogUpdates, best practices, user-stories

    • EventsWebinars, meetups, office hours

    • Success StoriesReal-world workload examples

    • EcosystemLibraries integrated with Ray

    • CommunityConnect with us

Try Ray on Anyscale
  • Overview
  • Getting Started
  • Installation
  • Use Cases
    • Ray for ML Infrastructure
  • Example Gallery
  • Ecosystem
  • Ray Core
    • Key Concepts
    • User Guides
      • Tasks
        • Nested Remote Functions
        • Dynamic generators
      • Actors
        • Named Actors
        • Terminating Actors
        • AsyncIO / Concurrency for Actors
        • Limiting Concurrency Per-Method with Concurrency Groups
        • Utility Classes
        • Out-of-band Communication
        • Actor Task Execution Order
      • Objects
        • Serialization
        • Object Spilling
      • Environment Dependencies
      • Scheduling
        • Resources
        • Accelerator Support
        • Placement Groups
        • Memory Management
        • Out-Of-Memory Prevention
      • Fault Tolerance
        • Task Fault Tolerance
        • Actor Fault Tolerance
        • Object Fault Tolerance
        • Node Fault Tolerance
        • GCS Fault Tolerance
      • Design Patterns & Anti-patterns
        • Pattern: Using nested tasks to achieve nested parallelism
        • Pattern: Using generators to reduce heap memory usage
        • Pattern: Using ray.wait to limit the number of pending tasks
        • Pattern: Using resources to limit the number of concurrently running tasks
        • Pattern: Using asyncio to run actor methods concurrently
        • Pattern: Using an actor to synchronize other tasks and actors
        • Pattern: Using a supervisor actor to manage a tree of actors
        • Pattern: Using pipelining to increase throughput
        • Anti-pattern: Returning ray.put() ObjectRefs from a task harms performance and fault tolerance
        • Anti-pattern: Calling ray.get in a loop harms parallelism
        • Anti-pattern: Calling ray.get unnecessarily harms performance
        • Anti-pattern: Processing results in submission order using ray.get increases runtime
        • Anti-pattern: Fetching too many objects at once with ray.get causes failure
        • Anti-pattern: Over-parallelizing with too fine-grained tasks harms speedup
        • Anti-pattern: Redefining the same remote function or class harms performance
        • Anti-pattern: Passing the same large argument by value repeatedly harms performance
        • Anti-pattern: Closure capturing large objects harms performance
        • Anti-pattern: Using global variables to share state between tasks and actors
        • Anti-pattern: Serialize ray.ObjectRef out of band
      • Advanced Topics
        • Tips for first-time users
        • Starting Ray
        • Ray Generators
        • Using Namespaces
        • Cross-Language Programming
        • Working with Jupyter Notebooks & JupyterLab
        • Lazy Computation Graphs with the Ray DAG API
        • Miscellaneous Topics
        • Authenticating Remote URIs in runtime_env
        • Lifetimes of a User-Spawn Process
    • Examples
      • Simple AutoML for time series with Ray Core
      • Batch Prediction with Ray Core
      • A Gentle Introduction to Ray Core by Example
      • Using Ray for Highly Parallelizable Tasks
      • A Simple MapReduce Example with Ray Core
      • Monte Carlo Estimation of π
      • Simple Parallel Model Selection
      • Parameter Server
      • Learning to Play Pong
      • Speed up your web crawler by parallelizing it with Ray
    • Ray Core API
      • Core API
      • Scheduling API
      • Runtime Env API
      • Utility
      • Exceptions
      • Ray Core CLI
      • State CLI
      • State API
  • Ray Data
    • Overview
    • Quickstart
    • User Guides
      • Loading Data
      • Inspecting Data
      • Transforming Data
      • Iterating over Data
      • Shuffling Data
      • Saving Data
      • Working with Images
      • Working with Text
      • Working with Tensors / NumPy
      • Working with PyTorch
      • Monitoring Your Workload
      • Execution Configurations
      • End-to-end: Offline Batch Inference
      • Advanced: Performance Tips and Tuning
      • Advanced: Read and Write Custom File Types
    • Examples
    • Ray Data API
      • Input/Output
      • Dataset API
      • DataIterator API
      • ExecutionOptions API
      • GroupedData API
      • Global configuration
      • Preprocessor
      • API Guide for Users from Other Data Libraries
    • Ray Data Internals
  • Ray Train
    • Overview
    • PyTorch Guide
    • PyTorch Lightning Guide
    • Hugging Face Transformers Guide
    • More Frameworks
      • Hugging Face Accelerate Guide
      • DeepSpeed Guide
      • TensorFlow and Keras Guide
      • XGBoost and LightGBM Guide
      • Horovod Guide
    • User Guides
      • Data Loading and Preprocessing
      • Configuring Scale and GPUs
      • Configuring Persistent Storage
      • Monitoring and Logging Metrics
      • Saving and Loading Checkpoints
      • Experiment Tracking
      • Inspecting Training Results
      • Handling Failures and Node Preemption
      • Reproducibility
      • Hyperparameter Optimization
    • Examples
    • Benchmarks
    • Ray Train API
  • Ray Tune
    • Getting Started
    • Key Concepts
    • User Guides
      • Running Basic Experiments
      • Logging and Outputs in Tune
      • Setting Trial Resources
      • Using Search Spaces
      • How to Define Stopping Criteria for a Ray Tune Experiment
      • How to Save and Load Trial Checkpoints
      • How to Configure Persistent Storage in Ray Tune
      • How to Enable Fault Tolerance in Ray Tune
      • Using Callbacks and Metrics
      • Getting Data in and out of Tune
      • Analyzing Tune Experiment Results
      • A Guide to Population Based Training with Tune
        • Visualizing and Understanding PBT
      • Deploying Tune in the Cloud
      • Tune Architecture
      • Scalability Benchmarks
    • Ray Tune Examples
      • Examples using Ray Tune with ML Frameworks
        • Keras Example
        • PyTorch Example
        • PyTorch Lightning Example
        • Ray RLlib Example
        • XGBoost Example
        • LightGBM Example
        • Horovod Example
        • Hugging Face Transformers Example
      • Tune Experiment Tracking Examples
        • Weights & Biases Example
        • MLflow Example
        • Aim Example
        • Comet Example
      • Tune Hyperparameter Optimization Framework Examples
        • Ax Example
        • HyperOpt Example
        • Bayesopt Example
        • BOHB Example
        • Nevergrad Example
        • Optuna Example
      • Other Examples
      • Exercises
    • Ray Tune FAQ
    • Ray Tune API
      • Tune Execution (tune.Tuner)
      • Tune Experiment Results (tune.ResultGrid)
      • Training in Tune (tune.Trainable, train.report)
      • Tune Search Space API
      • Tune Search Algorithms (tune.search)
      • Tune Trial Schedulers (tune.schedulers)
      • Tune Stopping Mechanisms (tune.stopper)
      • Tune Console Output (Reporters)
      • Syncing in Tune (train.SyncConfig)
      • Tune Loggers (tune.logger)
      • Tune Callbacks (tune.Callback)
      • Environment variables used by Ray Tune
      • External library integrations for Ray Tune
      • Tune Internals
      • Tune CLI (Experimental)
  • Ray Serve
    • Getting Started
    • Key Concepts
    • Develop and Deploy an ML Application
    • Deploy Compositions of Models
    • Deploy Multiple Applications
    • Model Multiplexing
    • Configure Ray Serve deployments
    • Set Up FastAPI and HTTP
    • Production Guide
      • Serve Config Files
      • Deploy on Kubernetes
      • Custom Docker Images
      • Add End-to-End Fault Tolerance
      • Handle Dependencies
      • Best practices in production
    • Monitor Your Application
    • Resource Allocation
    • Ray Serve Autoscaling
    • Advanced Guides
      • Pass Arguments to Applications
      • Advanced Ray Serve Autoscaling
      • Performance Tuning
      • Dynamic Request Batching
      • Updating Applications In-Place
      • Development Workflow
      • Set Up a gRPC Service
      • Experimental Java API
      • Deploy on VM
      • Run Multiple Applications in Different Containers
    • Architecture
    • Examples
    • Ray Serve API
  • Ray RLlib
    • Getting Started with RLlib
    • Key Concepts
    • Environments
    • Algorithms
    • User Guides
      • Advanced Python APIs
      • Models, Preprocessors, and Action Distributions
      • Saving and Loading your RL Algorithms and Policies
      • How To Customize Policies
      • Episodes
      • Replay Buffers
      • Working With Offline Data
      • Catalog (Alpha)
      • RL Modules (Alpha)
      • Learner (Alpha)
      • Using RLlib with torch 2.x compile
      • Fault Tolerance And Elastic Training
      • Install RLlib for Development
    • Examples
    • RLlib’s new API stack
    • New API stack migration guide
    • Ray RLlib API
      • Algorithms
        • ray.rllib.algorithms.algorithm_config.AlgorithmConfig
        • ray.rllib.algorithms.algorithm_config.AlgorithmConfig.copy
        • ray.rllib.algorithms.algorithm_config.AlgorithmConfig.validate
        • ray.rllib.algorithms.algorithm_config.AlgorithmConfig.freeze
        • ray.rllib.algorithms.algorithm_config.AlgorithmConfig.build
        • ray.rllib.algorithms.algorithm_config.AlgorithmConfig.build_learner_group
        • ray.rllib.algorithms.algorithm_config.AlgorithmConfig.build_learner
        • ray.rllib.algorithms.algorithm_config.AlgorithmConfig.callbacks
        • ray.rllib.algorithms.algorithm_config.AlgorithmConfig.debugging
        • ray.rllib.algorithms.algorithm_config.AlgorithmConfig.environment
        • ray.rllib.algorithms.algorithm_config.AlgorithmConfig.evaluation
        • ray.rllib.algorithms.algorithm_config.AlgorithmConfig.experimental
        • ray.rllib.algorithms.algorithm_config.AlgorithmConfig.fault_tolerance
        • ray.rllib.algorithms.algorithm_config.AlgorithmConfig.framework
        • ray.rllib.algorithms.algorithm_config.AlgorithmConfig.multi_agent
        • ray.rllib.algorithms.algorithm_config.AlgorithmConfig.offline_data
        • ray.rllib.algorithms.algorithm_config.AlgorithmConfig.python_environment
        • ray.rllib.algorithms.algorithm_config.AlgorithmConfig.reporting
        • ray.rllib.algorithms.algorithm_config.AlgorithmConfig.resources
        • ray.rllib.algorithms.algorithm_config.AlgorithmConfig.rl_module
        • ray.rllib.algorithms.algorithm_config.AlgorithmConfig.rollouts
        • ray.rllib.algorithms.algorithm_config.AlgorithmConfig.training
        • ray.rllib.algorithms.algorithm_config.AlgorithmConfig.get_default_learner_class
        • ray.rllib.algorithms.algorithm_config.AlgorithmConfig.get_default_rl_module_spec
        • ray.rllib.algorithms.algorithm_config.AlgorithmConfig.get_evaluation_config_object
        • ray.rllib.algorithms.algorithm_config.AlgorithmConfig.get_multi_rl_module_spec
        • ray.rllib.algorithms.algorithm_config.AlgorithmConfig.get_multi_agent_setup
        • ray.rllib.algorithms.algorithm_config.AlgorithmConfig.get_rollout_fragment_length
        • ray.rllib.algorithms.algorithm_config.AlgorithmConfig.validate_train_batch_size_vs_rollout_fragment_length
        • ray.rllib.algorithms.algorithm.Algorithm
        • ray.rllib.algorithms.algorithm.Algorithm.setup
        • ray.rllib.algorithms.algorithm.Algorithm.get_default_config
        • ray.rllib.algorithms.algorithm.Algorithm.compute_actions
        • ray.rllib.algorithms.algorithm.Algorithm.compute_single_action
        • ray.rllib.algorithms.algorithm.Algorithm.evaluate
        • ray.rllib.algorithms.algorithm.Algorithm.from_checkpoint
        • ray.rllib.algorithms.algorithm.Algorithm.from_state
        • ray.rllib.algorithms.algorithm.Algorithm.get_weights
        • ray.rllib.algorithms.algorithm.Algorithm.set_weights
        • ray.rllib.algorithms.algorithm.Algorithm.export_model
        • ray.rllib.algorithms.algorithm.Algorithm.export_policy_checkpoint
        • ray.rllib.algorithms.algorithm.Algorithm.export_policy_model
        • ray.rllib.algorithms.algorithm.Algorithm.restore
        • ray.rllib.algorithms.algorithm.Algorithm.restore_workers
        • ray.rllib.algorithms.algorithm.Algorithm.save
        • ray.rllib.algorithms.algorithm.Algorithm.save_checkpoint
        • ray.rllib.algorithms.algorithm.Algorithm.train
        • ray.rllib.algorithms.algorithm.Algorithm.training_step
        • ray.rllib.algorithms.algorithm.Algorithm.add_policy
        • ray.rllib.algorithms.algorithm.Algorithm.remove_policy
      • Environments
        • BaseEnv API
        • MultiAgentEnv API
        • VectorEnv API
        • ExternalEnv API
        • SingleAgentEpisode API
      • Policy API
        • ray.rllib.policy.policy.Policy
        • ray.rllib.policy.eager_tf_policy_v2.EagerTFPolicyV2
        • ray.rllib.policy.torch_policy_v2.TorchPolicyV2
        • ray.rllib.policy.torch_policy_v2.TorchPolicyV2.make_model
        • ray.rllib.policy.torch_policy_v2.TorchPolicyV2.make_model_and_action_dist
        • ray.rllib.policy.eager_tf_policy_v2.EagerTFPolicyV2.make_model
        • ray.rllib.policy.policy.Policy.compute_actions
        • ray.rllib.policy.policy.Policy.compute_actions_from_input_dict
        • ray.rllib.policy.policy.Policy.compute_single_action
        • ray.rllib.policy.torch_policy_v2.TorchPolicyV2.action_sampler_fn
        • ray.rllib.policy.torch_policy_v2.TorchPolicyV2.action_distribution_fn
        • ray.rllib.policy.torch_policy_v2.TorchPolicyV2.extra_action_out
        • ray.rllib.policy.eager_tf_policy_v2.EagerTFPolicyV2.action_sampler_fn
        • ray.rllib.policy.eager_tf_policy_v2.EagerTFPolicyV2.action_distribution_fn
        • ray.rllib.policy.eager_tf_policy_v2.EagerTFPolicyV2.extra_action_out_fn
        • ray.rllib.policy.Policy.compute_gradients
        • ray.rllib.policy.Policy.apply_gradients
        • ray.rllib.policy.torch_policy_v2.TorchPolicyV2.extra_compute_grad_fetches
        • ray.rllib.policy.torch_policy_v2.TorchPolicyV2.extra_grad_process
        • ray.rllib.policy.eager_tf_policy_v2.EagerTFPolicyV2.grad_stats_fn
        • ray.rllib.policy.eager_tf_policy_v2.EagerTFPolicyV2.compute_gradients_fn
        • ray.rllib.policy.eager_tf_policy_v2.EagerTFPolicyV2.apply_gradients_fn
        • ray.rllib.policy.eager_tf_policy_v2.EagerTFPolicyV2.extra_learn_fetches_fn
        • ray.rllib.policy.Policy.learn_on_batch
        • ray.rllib.policy.Policy.load_batch_into_buffer
        • ray.rllib.policy.Policy.learn_on_loaded_batch
        • ray.rllib.policy.Policy.learn_on_batch_from_replay_buffer
        • ray.rllib.policy.Policy.get_num_samples_loaded_into_buffer
        • ray.rllib.policy.Policy.loss
        • ray.rllib.policy.Policy.compute_log_likelihoods
        • ray.rllib.policy.Policy.on_global_var_update
        • ray.rllib.policy.Policy.postprocess_trajectory
        • ray.rllib.policy.torch_policy_v2.TorchPolicyV2.optimizer
        • ray.rllib.policy.torch_policy_v2.TorchPolicyV2.get_tower_stats
        • ray.rllib.policy.eager_tf_policy_v2.EagerTFPolicyV2.optimizer
        • ray.rllib.policy.eager_tf_policy_v2.EagerTFPolicyV2.stats_fn
        • ray.rllib.policy.Policy.from_checkpoint
        • ray.rllib.policy.Policy.export_checkpoint
        • ray.rllib.policy.Policy.export_model
        • ray.rllib.policy.Policy.from_state
        • ray.rllib.policy.Policy.get_weights
        • ray.rllib.policy.Policy.set_weights
        • ray.rllib.policy.Policy.get_state
        • ray.rllib.policy.Policy.set_state
        • ray.rllib.policy.Policy.import_model_from_h5
        • ray.rllib.Policy.get_initial_state
        • ray.rllib.Policy.num_state_tensors
        • ray.rllib.Policy.is_recurrent
        • ray.rllib.policy.Policy.apply
        • ray.rllib.policy.Policy.get_session
        • ray.rllib.policy.Policy.init_view_requirements
        • ray.rllib.policy.Policy.get_host
        • ray.rllib.policy.Policy.get_exploration_state
        • ray.rllib.policy.torch_policy_v2.TorchPolicyV2.get_batch_divisibility_req
        • ray.rllib.policy.eager_tf_policy_v2.EagerTFPolicyV2.variables
        • ray.rllib.policy.eager_tf_policy_v2.EagerTFPolicyV2.get_batch_divisibility_req
      • Model APIs
        • ray.rllib.models.modelv2.ModelV2
        • ray.rllib.models.torch.torch_modelv2.TorchModelV2
        • ray.rllib.models.tf.tf_modelv2.TFModelV2
        • ray.rllib.models.modelv2.ModelV2.forward
        • ray.rllib.models.modelv2.ModelV2.value_function
        • ray.rllib.models.modelv2.ModelV2.last_output
        • ray.rllib.models.modelv2.ModelV2.get_initial_state
        • ray.rllib.models.modelv2.ModelV2.is_time_major
        • ray.rllib.models.modelv2.ModelV2.variables
        • ray.rllib.models.modelv2.ModelV2.trainable_variables
        • ray.rllib.models.distributions.Distribution
        • ray.rllib.models.modelv2.ModelV2.custom_loss
        • ray.rllib.models.modelv2.ModelV2.metrics
      • Catalog API
        • ray.rllib.core.models.catalog.Catalog
        • ray.rllib.core.models.catalog.Catalog.build_encoder
        • ray.rllib.core.models.catalog.Catalog.get_action_dist_cls
        • ray.rllib.core.models.catalog.Catalog.get_tokenizer_config
        • ray.rllib.core.models.catalog.Catalog.latent_dims
        • ray.rllib.core.models.catalog.Catalog._determine_components_hook
        • ray.rllib.core.models.catalog.Catalog._get_encoder_config
        • ray.rllib.core.models.catalog.Catalog._get_dist_cls_from_action_space
      • RLModule API
        • ray.rllib.core.rl_module.rl_module.RLModuleSpec
        • ray.rllib.core.rl_module.rl_module.RLModuleSpec.build
        • ray.rllib.core.rl_module.rl_module.RLModuleSpec.get_rl_module_config
        • ray.rllib.core.rl_module.rl_module.RLModuleConfig
        • ray.rllib.core.rl_module.rl_module.RLModuleConfig.to_dict
        • ray.rllib.core.rl_module.rl_module.RLModuleConfig.from_dict
        • ray.rllib.core.rl_module.rl_module.RLModuleConfig.get_catalog
        • ray.rllib.core.rl_module.multi_rl_module.MultiRLModuleSpec
        • ray.rllib.core.rl_module.multi_rl_module.MultiRLModuleSpec.build
        • ray.rllib.core.rl_module.multi_rl_module.MultiRLModuleSpec.get_multi_rl_module_config
        • ray.rllib.core.rl_module.rl_module.RLModule
        • ray.rllib.core.rl_module.rl_module.RLModule.as_multi_rl_module
        • ray.rllib.core.rl_module.rl_module.RLModule.forward_train
        • ray.rllib.core.rl_module.rl_module.RLModule.forward_exploration
        • ray.rllib.core.rl_module.rl_module.RLModule.forward_inference
        • ray.rllib.core.rl_module.rl_module.RLModule._forward_train
        • ray.rllib.core.rl_module.rl_module.RLModule._forward_exploration
        • ray.rllib.core.rl_module.rl_module.RLModule._forward_inference
        • ray.rllib.core.rl_module.rl_module.RLModule.input_specs_inference
        • ray.rllib.core.rl_module.rl_module.RLModule.input_specs_exploration
        • ray.rllib.core.rl_module.rl_module.RLModule.input_specs_train
        • ray.rllib.core.rl_module.rl_module.RLModule.output_specs_inference
        • ray.rllib.core.rl_module.rl_module.RLModule.output_specs_exploration
        • ray.rllib.core.rl_module.rl_module.RLModule.output_specs_train
        • ray.rllib.core.rl_module.rl_module.RLModule.get_state
        • ray.rllib.core.rl_module.rl_module.RLModule.set_state
        • ray.rllib.core.rl_module.rl_module.RLModule.save_to_path
        • ray.rllib.core.rl_module.rl_module.RLModule.restore_from_path
        • ray.rllib.core.rl_module.rl_module.RLModule.from_checkpoint
        • ray.rllib.core.rl_module.multi_rl_module.MultiRLModule
        • ray.rllib.core.rl_module.multi_rl_module.MultiRLModule.setup
        • ray.rllib.core.rl_module.multi_rl_module.MultiRLModule.as_multi_rl_module
        • ray.rllib.core.rl_module.multi_rl_module.MultiRLModule.add_module
        • ray.rllib.core.rl_module.multi_rl_module.MultiRLModule.remove_module
        • ray.rllib.core.rl_module.multi_rl_module.MultiRLModule.save_to_path
        • ray.rllib.core.rl_module.multi_rl_module.MultiRLModule.restore_from_path
      • LearnerGroup API
        • ray.rllib.algorithms.algorithm_config.AlgorithmConfig.resources
        • ray.rllib.algorithms.algorithm_config.AlgorithmConfig.rl_module
        • ray.rllib.algorithms.algorithm_config.AlgorithmConfig.training
        • ray.rllib.algorithms.algorithm_config.AlgorithmConfig.build_learner_group
        • ray.rllib.core.learner.learner_group.LearnerGroup
      • Sampling the Environment or offline data
        • ray.rllib.evaluation.rollout_worker.RolloutWorker
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.add_policy
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.remove_policy
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.get_policy
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.set_is_policy_to_train
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.set_policy_mapping_fn
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.for_policy
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.foreach_policy
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.foreach_policy_to_train
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.get_filters
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.get_global_vars
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.set_global_vars
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.get_host
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.get_metrics
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.get_node_ip
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.get_weights
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.set_weights
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.get_state
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.set_state
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.lock
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.unlock
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.sample
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.sample_with_count
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.sample_and_learn
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.learn_on_batch
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.setup_torch_data_parallel
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.compute_gradients
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.apply_gradients
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.foreach_env
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.foreach_env_with_context
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.stop
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.apply
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.sync_filters
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.find_free_port
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.creation_args
        • ray.rllib.evaluation.rollout_worker.RolloutWorker.assert_healthy
        • ray.rllib.env.env_runner.EnvRunner
        • ray.rllib.env.env_runner_group.EnvRunnerGroup
        • ray.rllib.env.env_runner_group.EnvRunnerGroup.stop
        • ray.rllib.env.env_runner_group.EnvRunnerGroup.reset
        • ray.rllib.env.env_runner_group.EnvRunnerGroup.add_workers
        • ray.rllib.env.env_runner_group.EnvRunnerGroup.foreach_worker
        • ray.rllib.env.env_runner_group.EnvRunnerGroup.foreach_worker_with_id
        • ray.rllib.env.env_runner_group.EnvRunnerGroup.foreach_worker_async
        • ray.rllib.env.env_runner_group.EnvRunnerGroup.fetch_ready_async_reqs
        • ray.rllib.env.env_runner_group.EnvRunnerGroup.num_in_flight_async_reqs
        • ray.rllib.env.env_runner_group.EnvRunnerGroup.local_worker
        • ray.rllib.env.env_runner_group.EnvRunnerGroup.remote_workers
        • ray.rllib.env.env_runner_group.EnvRunnerGroup.num_healthy_remote_workers
        • ray.rllib.env.env_runner_group.EnvRunnerGroup.num_healthy_workers
        • ray.rllib.env.env_runner_group.EnvRunnerGroup.num_remote_worker_restarts
        • ray.rllib.env.env_runner_group.EnvRunnerGroup.probe_unhealthy_workers
        • ray.rllib.env.env_runner_group.EnvRunnerGroup.add_policy
        • ray.rllib.env.env_runner_group.EnvRunnerGroup.foreach_env
        • ray.rllib.env.env_runner_group.EnvRunnerGroup.foreach_env_with_context
        • ray.rllib.env.env_runner_group.EnvRunnerGroup.foreach_policy
        • ray.rllib.env.env_runner_group.EnvRunnerGroup.foreach_policy_to_train
        • ray.rllib.env.env_runner_group.EnvRunnerGroup.sync_weights
        • ray.rllib.offline.input_reader.InputReader
        • ray.rllib.offline.input_reader.InputReader.next
        • ray.rllib.evaluation.sampler.SamplerInput
        • ray.rllib.evaluation.sampler.SamplerInput.get_data
        • ray.rllib.evaluation.sampler.SamplerInput.get_extra_batches
        • ray.rllib.evaluation.sampler.SamplerInput.get_metrics
        • ray.rllib.evaluation.sampler.SyncSampler
        • ray.rllib.offline.json_reader.JsonReader
        • ray.rllib.offline.json_reader.JsonReader.read_all_files
        • ray.rllib.offline.mixed_input.MixedInput
        • ray.rllib.offline.d4rl_reader.D4RLReader
        • ray.rllib.offline.io_context.IOContext
        • ray.rllib.offline.io_context.IOContext.default_sampler_input
        • ray.rllib.policy.policy_map.PolicyMap
        • ray.rllib.policy.policy_map.PolicyMap.items
        • ray.rllib.policy.policy_map.PolicyMap.keys
        • ray.rllib.policy.policy_map.PolicyMap.values
        • ray.rllib.policy.sample_batch.SampleBatch
        • ray.rllib.policy.sample_batch.SampleBatch.set_get_interceptor
        • ray.rllib.policy.sample_batch.SampleBatch.is_training
        • ray.rllib.policy.sample_batch.SampleBatch.set_training
        • ray.rllib.policy.sample_batch.SampleBatch.as_multi_agent
        • ray.rllib.policy.sample_batch.SampleBatch.get
        • ray.rllib.policy.sample_batch.SampleBatch.to_device
        • ray.rllib.policy.sample_batch.SampleBatch.right_zero_pad
        • ray.rllib.policy.sample_batch.SampleBatch.slice
        • ray.rllib.policy.sample_batch.SampleBatch.split_by_episode
        • ray.rllib.policy.sample_batch.SampleBatch.shuffle
        • ray.rllib.policy.sample_batch.SampleBatch.columns
        • ray.rllib.policy.sample_batch.SampleBatch.rows
        • ray.rllib.policy.sample_batch.SampleBatch.copy
        • ray.rllib.policy.sample_batch.SampleBatch.is_single_trajectory
        • ray.rllib.policy.sample_batch.SampleBatch.is_terminated_or_truncated
        • ray.rllib.policy.sample_batch.SampleBatch.env_steps
        • ray.rllib.policy.sample_batch.SampleBatch.agent_steps
        • ray.rllib.policy.sample_batch.MultiAgentBatch
        • ray.rllib.policy.sample_batch.MultiAgentBatch.env_steps
        • ray.rllib.policy.sample_batch.MultiAgentBatch.agent_steps
      • Replay Buffer API
        • ray.rllib.utils.replay_buffers.replay_buffer.StorageUnit
        • ray.rllib.utils.replay_buffers.replay_buffer.ReplayBuffer
        • ray.rllib.utils.replay_buffers.prioritized_replay_buffer.PrioritizedReplayBuffer
        • ray.rllib.utils.replay_buffers.reservoir_replay_buffer.ReservoirReplayBuffer
        • ray.rllib.utils.replay_buffers.replay_buffer.ReplayBuffer.sample
        • ray.rllib.utils.replay_buffers.replay_buffer.ReplayBuffer.add
        • ray.rllib.utils.replay_buffers.replay_buffer.ReplayBuffer.get_state
        • ray.rllib.utils.replay_buffers.replay_buffer.ReplayBuffer.set_state
        • ray.rllib.utils.replay_buffers.multi_agent_replay_buffer.MultiAgentReplayBuffer
        • ray.rllib.utils.replay_buffers.multi_agent_prioritized_replay_buffer.MultiAgentPrioritizedReplayBuffer
        • ray.rllib.utils.replay_buffers.utils.update_priorities_in_replay_buffer
        • ray.rllib.utils.replay_buffers.utils.sample_min_n_steps_from_buffer
      • RLlib Utilities
        • ray.rllib.utils.exploration.exploration.Exploration
        • ray.rllib.utils.exploration.random.Random
        • ray.rllib.utils.exploration.stochastic_sampling.StochasticSampling
        • ray.rllib.utils.exploration.epsilon_greedy.EpsilonGreedy
        • ray.rllib.utils.exploration.gaussian_noise.GaussianNoise
        • ray.rllib.utils.exploration.ornstein_uhlenbeck_noise.OrnsteinUhlenbeckNoise
        • ray.rllib.utils.exploration.random_encoder.RE3
        • ray.rllib.utils.exploration.curiosity.Curiosity
        • ray.rllib.utils.exploration.parameter_noise.ParameterNoise
        • ray.rllib.utils.exploration.exploration.Exploration.get_exploration_action
        • ray.rllib.utils.exploration.exploration.Exploration.before_compute_actions
        • ray.rllib.utils.exploration.exploration.Exploration.on_episode_start
        • ray.rllib.utils.exploration.exploration.Exploration.on_episode_end
        • ray.rllib.utils.exploration.exploration.Exploration.postprocess_trajectory
        • ray.rllib.utils.exploration.exploration.Exploration.get_state
        • ray.rllib.utils.exploration.exploration.Exploration.set_state
        • ray.rllib.utils.schedules.schedule.Schedule
        • ray.rllib.utils.schedules.constant_schedule.ConstantSchedule
        • ray.rllib.utils.schedules.linear_schedule.LinearSchedule
        • ray.rllib.utils.schedules.piecewise_schedule.PiecewiseSchedule
        • ray.rllib.utils.schedules.exponential_schedule.ExponentialSchedule
        • ray.rllib.utils.schedules.polynomial_schedule.PolynomialSchedule
        • ray.rllib.utils.schedules.schedule.Schedule.value
        • ray.rllib.utils.schedules.schedule.Schedule.__call__
        • ray.rllib.execution.train_ops.multi_gpu_train_one_step
        • ray.rllib.execution.train_ops.train_one_step
        • ray.rllib.utils.framework.try_import_torch
        • ray.rllib.utils.framework.try_import_tf
        • ray.rllib.utils.framework.try_import_tfp
        • ray.rllib.utils.tf_utils.explained_variance
        • ray.rllib.utils.tf_utils.flatten_inputs_to_1d_tensor
        • ray.rllib.utils.tf_utils.get_gpu_devices
        • ray.rllib.utils.tf_utils.get_placeholder
        • ray.rllib.utils.tf_utils.huber_loss
        • ray.rllib.utils.tf_utils.l2_loss
        • ray.rllib.utils.tf_utils.make_tf_callable
        • ray.rllib.utils.tf_utils.minimize_and_clip
        • ray.rllib.utils.tf_utils.one_hot
        • ray.rllib.utils.tf_utils.reduce_mean_ignore_inf
        • ray.rllib.utils.tf_utils.scope_vars
        • ray.rllib.utils.tf_utils.warn_if_infinite_kl_divergence
        • ray.rllib.utils.tf_utils.zero_logps_from_actions
        • ray.rllib.utils.torch_utils.apply_grad_clipping
        • ray.rllib.utils.torch_utils.concat_multi_gpu_td_errors
        • ray.rllib.utils.torch_utils.convert_to_torch_tensor
        • ray.rllib.utils.torch_utils.explained_variance
        • ray.rllib.utils.torch_utils.flatten_inputs_to_1d_tensor
        • ray.rllib.utils.torch_utils.global_norm
        • ray.rllib.utils.torch_utils.huber_loss
        • ray.rllib.utils.torch_utils.l2_loss
        • ray.rllib.utils.torch_utils.minimize_and_clip
        • ray.rllib.utils.torch_utils.one_hot
        • ray.rllib.utils.torch_utils.reduce_mean_ignore_inf
        • ray.rllib.utils.torch_utils.sequence_mask
        • ray.rllib.utils.torch_utils.warn_if_infinite_kl_divergence
        • ray.rllib.utils.torch_utils.set_torch_seed
        • ray.rllib.utils.torch_utils.softmax_cross_entropy_with_logits
        • ray.rllib.utils.numpy.aligned_array
        • ray.rllib.utils.numpy.concat_aligned
        • ray.rllib.utils.numpy.convert_to_numpy
        • ray.rllib.utils.numpy.fc
        • ray.rllib.utils.numpy.flatten_inputs_to_1d_tensor
        • ray.rllib.utils.numpy.make_action_immutable
        • ray.rllib.utils.numpy.huber_loss
        • ray.rllib.utils.numpy.l2_loss
        • ray.rllib.utils.numpy.lstm
        • ray.rllib.utils.numpy.one_hot
        • ray.rllib.utils.numpy.relu
        • ray.rllib.utils.numpy.sigmoid
        • ray.rllib.utils.numpy.softmax
        • ray.rllib.utils.checkpoints.Checkpointable
        • ray.rllib.utils.checkpoints.convert_to_msgpack_checkpoint
        • ray.rllib.utils.checkpoints.convert_to_msgpack_policy_checkpoint
        • ray.rllib.utils.checkpoints.get_checkpoint_info
        • ray.rllib.utils.checkpoints.try_import_msgpack
        • ray.rllib.utils.policy.compute_log_likelihoods_from_input_dict
        • ray.rllib.utils.policy.create_policy_for_framework
        • ray.rllib.utils.policy.local_policy_inference
        • ray.rllib.utils.policy.parse_policy_specs_from_checkpoint
        • ray.rllib.utils.tensor_dtype.get_np_dtype
        • ray.rllib.common.CLIArguments
        • ray.rllib.common.FrameworkEnum
        • ray.rllib.common.SupportedFileType
        • ray.rllib.core.rl_module.validate_module_id
        • ray.rllib.train.load_experiments_from_file
      • External Application API
  • More Libraries
    • Distributed Scikit-learn / Joblib
    • Distributed multiprocessing.Pool
    • Ray Collective Communication Lib
    • Using Dask on Ray
      • ray.util.dask.RayDaskCallback
        • ray.util.dask.RayDaskCallback.ray_active
      • ray.util.dask.callbacks.RayDaskCallback._ray_presubmit
      • ray.util.dask.callbacks.RayDaskCallback._ray_postsubmit
      • ray.util.dask.callbacks.RayDaskCallback._ray_pretask
      • ray.util.dask.callbacks.RayDaskCallback._ray_posttask
      • ray.util.dask.callbacks.RayDaskCallback._ray_postsubmit_all
      • ray.util.dask.callbacks.RayDaskCallback._ray_finish
    • Using Spark on Ray (RayDP)
    • Using Mars on Ray
    • Using Pandas on Ray (Modin)
    • Ray Workflows (Alpha)
      • Key Concepts
      • Getting Started
      • Workflow Management
      • Workflow Metadata
      • Events
      • API Comparisons
      • Advanced Topics
      • Ray Workflows API
        • Workflow Execution API
        • Workflow Management API
  • Ray Clusters
    • Key Concepts
    • Deploying on Kubernetes
      • Getting Started with KubeRay
        • RayCluster Quickstart
        • RayJob Quickstart
        • RayService Quickstart
      • User Guides
        • Deploy Ray Serve Apps
        • RayService high availability
        • KubeRay Observability
        • KubeRay upgrade guide
        • Managed Kubernetes services
        • Best Practices for Storage and Dependencies
        • RayCluster Configuration
        • KubeRay Autoscaling
        • GCS fault tolerance in KubeRay
        • Configuring KubeRay to use Google Cloud Storage Buckets in GKE
        • Log Persistence
        • Using GPUs
        • Use TPUs with KubeRay
        • Developing Ray Serve Python scripts on a RayCluster
        • Specify container commands for Ray head/worker Pods
        • Pod Security
        • Helm Chart RBAC
        • TLS Authentication
        • (Advanced) Understanding the Ray Autoscaler in the Context of Kubernetes
        • (Advanced) Deploying a static Ray cluster without KubeRay
        • Use kubectl Plugin (alpha)
      • Examples
        • Ray Train XGBoostTrainer on Kubernetes
        • Train PyTorch ResNet model with GPUs on Kubernetes
        • Train a PyTorch model on Fashion MNIST with CPUs on Kubernetes
        • Serve a StableDiffusion text-to-image model on Kubernetes
        • Serve a Stable Diffusion model on GKE with TPUs
        • Serve a MobileNet image classifier on Kubernetes
        • Serve a text summarizer on Kubernetes
        • RayJob Batch Inference Example
        • Priority Scheduling with RayJob and Kueue
        • Gang Scheduling with RayJob and Kueue
        • Distributed checkpointing with KubeRay and GCSFuse
        • Use Modin with Ray on Kubernetes
        • Serve a Large Language Model with vLLM on Kubernetes
      • KubeRay Ecosystem
        • Ingress
        • Using Prometheus and Grafana
        • Profiling with py-spy
        • KubeRay integration with Volcano
        • KubeRay integration with Apache YuniKorn
        • Kubeflow: an interactive development solution
        • Gang scheduling and priority scheduling for RayJob with Kueue
        • mTLS and L7 observability with Istio
      • KubeRay Benchmarks
        • KubeRay memory and scalability benchmark
      • KubeRay Troubleshooting
        • Troubleshooting guide
        • RayService troubleshooting
      • API Reference
    • Deploying on VMs
      • Getting Started
      • User Guides
        • Launching Ray Clusters on AWS, GCP, Azure, vSphere, On-Prem
        • Best practices for deploying large clusters
        • Configuring Autoscaling
        • Log Persistence
        • Community Supported Cluster Managers
      • Examples
        • Ray Train XGBoostTrainer on VMs
      • API References
        • Cluster Launcher Commands
        • Cluster YAML Configuration Options
    • Collecting and monitoring metrics
    • Configuring and Managing Ray Dashboard
    • Applications Guide
      • Ray Jobs Overview
        • Quickstart using the Ray Jobs CLI
        • Python SDK Overview
        • Python SDK API Reference
        • Ray Jobs CLI API Reference
        • Ray Jobs REST API
        • Ray Client
      • Programmatic Cluster Scaling
    • FAQ
    • Ray Cluster Management API
      • Cluster Management CLI
      • Python SDK API Reference
      • Ray Jobs CLI API Reference
      • Programmatic Cluster Scaling
    • Usage Stats Collection
  • Monitoring and Debugging
    • Ray Dashboard
    • Ray Distributed Debugger
    • Key Concepts
    • User Guides
      • Debugging Applications
        • General Debugging
        • Debugging Memory Issues
        • Debugging Hangs
        • Debugging Failures
        • Optimizing Performance
        • Ray Distributed Debugger
        • Using the Ray Debugger
      • Monitoring with the CLI or SDK
      • Configuring Logging
      • Profiling
      • Adding Application-Level Metrics
      • Tracing
    • Reference
      • State API
      • State CLI
      • System Metrics
  • Developer Guides
    • API Stability
    • API Policy
    • Getting Involved / Contributing
      • Building Ray from Source
      • CI Testing Workflow on PRs
      • Contributing to the Ray Documentation
      • How to write code snippets
      • Testing Autoscaling Locally
      • Tips for testing Ray programs
      • Debugging for Ray Developers
      • Profiling for Ray Developers
    • Configuring Ray
    • Architecture Whitepapers
  • Glossary
  • Security
  • RLlib: Industry-Grade, Scalable Reinforcement Learning
  • ray.rllib.core.rl_module.multi_rl_module.MultiRLModuleSpec
  • ray.rllib.co...

ray.rllib.core.rl_module.multi_rl_module.MultiRLModuleSpec.module_specs#

MultiRLModuleSpec.module_specs: RLModuleSpec | Dict[str, RLModuleSpec] | None = None#

previous

ray.rllib.core.rl_module.multi_rl_module.MultiRLModuleSpec.model_config

next

ray.rllib.core.rl_module.multi_rl_module.MultiRLModuleSpec.modules_to_load
On this page
  • MultiRLModuleSpec.module_specs
Thanks for the feedback!
Was this helpful?
Yes
No
Feedback
Submit

© Copyright 2024, The Ray Team.

Created using Sphinx 7.3.7.

Built with the PyData Sphinx Theme 0.14.1.