Skip to main content
Ctrl+K
Try Ray with $100 credit — Start now

Site Navigation

  • Get Started

  • Use Cases

  • Example Gallery

  • Library

    • Ray CoreScale general Python applications

    • Ray DataScale data ingest and preprocessing

    • Ray TrainScale machine learning training

    • Ray TuneScale hyperparameter tuning

    • Ray ServeScale model serving

    • Ray RLlibScale reinforcement learning

  • APIs

  • Resources

    • Discussion ForumGet your Ray questions answered

    • TrainingHands-on learning

    • BlogUpdates, best practices, user-stories

    • EventsWebinars, meetups, office hours

    • Success StoriesReal-world workload examples

    • EcosystemLibraries integrated with Ray

    • CommunityConnect with us

Try Managed Ray

Site Navigation

  • Get Started

  • Use Cases

  • Example Gallery

  • Library

    • Ray CoreScale general Python applications

    • Ray DataScale data ingest and preprocessing

    • Ray TrainScale machine learning training

    • Ray TuneScale hyperparameter tuning

    • Ray ServeScale model serving

    • Ray RLlibScale reinforcement learning

  • APIs

  • Resources

    • Discussion ForumGet your Ray questions answered

    • TrainingHands-on learning

    • BlogUpdates, best practices, user-stories

    • EventsWebinars, meetups, office hours

    • Success StoriesReal-world workload examples

    • EcosystemLibraries integrated with Ray

    • CommunityConnect with us

Try Managed Ray
  • Overview
  • Getting Started
  • Installation
  • Use Cases
    • Ray for ML Infrastructure
  • Examples
    • Multi-modal AI pipeline
      • Batch inference
      • Distributed training
      • Online serving
    • LLM training and inference
    • Audio batch inference
    • Distributed XGBoost pipeline
      • Distributed training of an XGBoost model
      • Model validation using offline batch inference
      • Scalable online XGBoost inference with Ray Serve
    • Time-series forecasting
      • Distributed training of a DLinear time-series model
      • DLinear model validation using offline batch inference
      • Online serving for DLinear model using Ray Serve
    • Scalable video processing
      • Fine-tuning a face mask detection model with Faster R-CNN
      • Object detection batch inference on test dataset and metrics calculation
      • Video processing with object detection using batch inference
      • Host an object detection model as a service
    • Distributed RAG pipeline
      • Build a Regular RAG Document Ingestion Pipeline (No Ray required)
      • Scalable RAG Data Ingestion and Pagination with Ray Data
      • Deploy LLM with Ray Serve LLM
      • Build Basic RAG App
      • Improve RAG with Prompt Engineering
      • Evaluate RAG with Online Inference
      • Evaluate RAG using Batch Inference with Ray Data LLM
    • Deploy MCP servers
      • Deploying a custom MCP in Streamable HTTP mode with Ray Serve
      • Deploy an MCP Gateway with existing Ray Serve apps
      • Deploying an MCP STDIO Server as a scalable HTTP service with Ray Serve
      • Deploying multiple MCP services with Ray Serve
      • Build a Docker image for an MCP server
    • Build a tool-using agent
    • Build a multi-agent system with the A2A protocol
  • Ecosystem
  • Ray Core
    • Key Concepts
    • User Guides
      • Tasks
        • Nested Remote Functions
      • Actors
        • Named Actors
        • Terminating Actors
        • AsyncIO / Concurrency for Actors
        • Limiting Concurrency Per-Method with Concurrency Groups
        • Utility Classes
        • Out-of-band Communication
        • Actor Task Execution Order
      • Objects
        • Serialization
        • Object Spilling
      • Environment Dependencies
      • Scheduling
        • Use labels to control scheduling
        • Resources
        • Accelerator Support
        • Placement Groups
        • Memory Management
        • Out-Of-Memory Prevention
      • Fault tolerance
        • Task Fault Tolerance
        • Actor Fault Tolerance
        • Object Fault Tolerance
        • Node Fault Tolerance
        • GCS Fault Tolerance
      • Design Patterns & Anti-patterns
        • Pattern: Using nested tasks to achieve nested parallelism
        • Pattern: Using generators to reduce heap memory usage
        • Pattern: Using ray.wait to limit the number of pending tasks
        • Pattern: Using resources to limit the number of concurrently running tasks
        • Pattern: Using asyncio to run actor methods concurrently
        • Pattern: Using an actor to synchronize other tasks and actors
        • Pattern: Using a supervisor actor to manage a tree of actors
        • Pattern: Using pipelining to increase throughput
        • Anti-pattern: Returning ray.put() ObjectRefs from a task harms performance and fault tolerance
        • Anti-pattern: Calling ray.get on task arguments harms performance
        • Anti-pattern: Calling ray.get in a loop harms parallelism
        • Anti-pattern: Calling ray.get unnecessarily harms performance
        • Anti-pattern: Processing results in submission order using ray.get increases runtime
        • Anti-pattern: Fetching too many objects at once with ray.get causes failure
        • Anti-pattern: Over-parallelizing with too fine-grained tasks harms speedup
        • Anti-pattern: Redefining the same remote function or class harms performance
        • Anti-pattern: Passing the same large argument by value repeatedly harms performance
        • Anti-pattern: Closure capturing large objects harms performance
        • Anti-pattern: Using global variables to share state between tasks and actors
        • Anti-pattern: Serialize ray.ObjectRef out of band
        • Anti-pattern: Forking new processes in application code
      • Ray Direct Transport (RDT)
        • Implementing a custom tensor transport (Advanced)
      • Ray Compiled Graph (beta)
        • Quickstart
        • Profiling
        • Experimental: Overlapping communication and computation
        • Troubleshooting
        • Compiled Graph API
      • Resource Isolation With Cgroup v2
      • Advanced topics
        • Tips for first-time users
        • Type hints in Ray
        • Starting Ray
        • Ray Generators
        • Using Namespaces
        • Cross-language programming
        • Working with Jupyter Notebooks & JupyterLab
        • Lazy Computation Graphs with the Ray DAG API
        • Miscellaneous Topics
        • Authenticating Remote URIs in runtime_env
        • Lifetimes of a User-Spawn Process
        • Head Node Memory Management
    • Examples
      • Batch Prediction with Ray Core
      • A Gentle Introduction to Ray Core by Example
      • Using Ray for Highly Parallelizable Tasks
      • A Simple MapReduce Example with Ray Core
      • Monte Carlo Estimation of π
      • Simple Parallel Model Selection
      • Parameter Server
      • Learning to Play Pong
      • Speed up your web crawler by parallelizing it with Ray
    • Internals
      • Task Lifecycle
      • Streaming Generator
      • Autoscaler v2
      • RPC Fault Tolerance
      • Token Authentication
      • Metric Exporter Infrastructure
      • Ray Event Exporter Infrastructure
      • Port Service Discovery
      • Object Spilling
  • Ray Data
    • Ray Data Quickstart
    • Key Concepts
    • User Guides
      • Loading Data
      • Inspecting Data
      • Transforming Data
      • Aggregating Data
      • Iterating over Data
      • Joining Data
      • Shuffling Data
      • Weighted Dataset Mixing
      • Saving Data
      • Working with Images
      • Working with Text
      • Working with Tensors / NumPy
      • Working with PyTorch
      • Working with LLMs
      • How to avoid out-of-memory errors (OOMs)
      • Monitoring Your Workload
      • Execution Configurations
      • Run multiple Datasets in one cluster
      • End-to-end: Offline Batch Inference
      • Advanced: Performance Tips and Tuning
      • Advanced: Scaling out expensive collate functions
      • Advanced: Read and Write Custom File Types
    • Examples
    • Contributing to Ray Data
      • Contributing Guide
      • How to write tests
    • Comparing Ray Data to other systems
    • Ray Data Benchmarks
    • Ray Data Internals
  • Ray Train
    • Overview
    • PyTorch Guide
    • PyTorch Lightning Guide
    • Hugging Face Transformers Guide
    • XGBoost Guide
    • JAX Guide
    • More Frameworks
      • Hugging Face Accelerate Guide
      • DeepSpeed Guide
      • TensorFlow and Keras Guide
      • LightGBM Guide
      • Horovod Guide
    • User Guides
      • Data Loading and Preprocessing
      • Configuring Scale and Accelerators
      • Configuring Persistent Storage
      • Monitoring and Logging Metrics
      • Saving and Loading Checkpoints
      • Validating checkpoints asynchronously
      • Experiment Tracking
      • Inspecting Training Results
      • Handling Failures and Node Preemption
      • Elastic training
      • Ray Train Metrics
      • Local Mode
      • Reproducibility
      • Hyperparameter Optimization
    • Tutorials
      • Introduction to Ray Train workloads
      • Computer vision pattern
      • Tabular workload pattern
      • Time series workload pattern
      • Generative computer vision pattern
      • Diffusion policy pattern
      • Recommendation system pattern
    • Examples
    • Benchmarks
  • Ray Tune
    • Getting Started
    • Key Concepts
    • User Guides
      • Running Basic Experiments
      • Logging and Outputs in Tune
      • Setting Trial Resources
      • Using Search Spaces
      • How to Define Stopping Criteria for a Ray Tune Experiment
      • How to Save and Load Trial Checkpoints
      • How to Configure Persistent Storage in Ray Tune
      • How to Enable Fault Tolerance in Ray Tune
      • Using Callbacks and Metrics
      • Getting Data in and out of Tune
      • Analyzing Tune Experiment Results
      • A Guide to Population Based Training with Tune
        • Visualizing and Understanding PBT
      • Deploying Tune in the Cloud
      • Tune Architecture
      • Scalability Benchmarks
    • Ray Tune Examples
      • PyTorch Example
      • PyTorch Lightning Example
      • XGBoost Example
      • LightGBM Example
      • Hugging Face Transformers Example
      • Ray RLlib Example
      • Keras Example
      • PyTorch with ASHA
      • Weights & Biases Example
      • MLflow Example
      • Aim Example
      • Comet Example
      • Ax Example
      • HyperOpt Example
      • Bayesopt Example
      • BOHB Example
      • Nevergrad Example
      • Optuna Example
    • Ray Tune FAQ
  • Ray Serve
    • Getting Started
    • Key Concepts
    • Develop and Deploy an ML Application
    • Deploy Compositions of Models
    • Deploy Multiple Applications
    • Model Multiplexing
    • Model Registry Integration
    • Configure Ray Serve deployments
    • Set Up FastAPI and HTTP
    • Serving LLMs
      • Quickstart
      • Examples
        • Deploy a small-sized LLM
        • Deploy a medium-sized LLM
        • Deploy a large-sized LLM
        • Deploy a vision LLM
        • Deploy a reasoning LLM
        • Deploy a hybrid reasoning LLM
        • Deploy gpt-oss
      • User Guides
        • Configuration reference
        • Deployment initialization
        • Multi-LoRA deployment
        • Cross-node parallelism
        • Data parallel attention
        • Fractional GPU serving
        • Prefill/decode disaggregation
        • KV cache offloading
        • Prefix-aware routing
        • Direct streaming
        • vLLM compatibility
        • SGLang integration
        • Observability and monitoring
      • Architecture
        • Architecture overview
        • Core components
        • Serving patterns
        • Request routing
      • Benchmarks
      • Troubleshooting
    • Production Guide
      • Serve Config Files
      • Deploy on Kubernetes
      • Custom Docker Images
      • Add End-to-End Fault Tolerance
      • Handle Dependencies
      • Best practices in production
    • Monitor Your Application
    • Resource Allocation
    • Ray Serve Autoscaling
    • Asynchronous Inference
    • Advanced Guides
      • Pass Arguments to Applications
      • Advanced Ray Serve Autoscaling
      • Asyncio and concurrency best practices in Ray Serve
      • Performance Tuning
      • Dynamic Request Batching
      • Updating Applications In-Place
      • Development Workflow
      • Set Up a gRPC Service
      • Replica ranks
      • Replica scheduling
      • Gang scheduling
      • Experimental Java API
      • Deploy on VM
      • Run Multiple Applications in Different Containers
      • Use Custom Algorithm for Request Routing
      • Use deployment-scoped actors
      • Troubleshoot multi-node GPU serving on KubeRay
    • Architecture
    • Examples
  • Ray RLlib
    • Getting Started
    • Key concepts
    • Environments
      • Multi-Agent Environments
      • Hierarchical Environments
      • External Environments and Applications
    • AlgorithmConfig API
    • Algorithms
    • User Guides
      • Advanced Python APIs
      • Callbacks
      • Checkpointing
      • MetricsLogger API
      • Episodes
      • ConnectorV2 and ConnectorV2 pipelines
        • Env-to-module pipelines
        • Learner connector pipelines
      • Replay Buffers
      • Working with offline data
      • RL Modules
      • Learner (Alpha)
      • Fault Tolerance And Elastic Training
      • Install RLlib for Development
      • RLlib scaling guide
    • Examples
    • New API stack migration guide
  • More Libraries
    • Distributed Scikit-learn / Joblib
    • Distributed multiprocessing.Pool
    • Ray Collective Communication Lib
      • Custom Collective Backends
    • Using Dask on Ray
      • RayDaskCallback
        • ray_active
      • _ray_presubmit
      • _ray_postsubmit
      • _ray_pretask
      • _ray_posttask
      • _ray_postsubmit_all
      • _ray_finish
    • Using Spark on Ray (RayDP)
    • Using Mars on Ray
    • Using Pandas on Ray (Modin)
    • Distributed Data Processing in Data-Juicer
  • APIs
    • Ray Data
      • Loading Data API
      • Saving Data API
      • Dataset API
      • DataIterator API
      • ExecutionOptions API
      • Checkpoint API
      • Aggregation API
      • GroupedData API
      • Expressions API
      • Data types
      • Global configuration
      • Preprocessor
      • Large Language Model (LLM) API
      • API Guide for Users from Other Data Libraries
    • Ray Train
    • Ray Tune
      • Tune Execution (tune.Tuner)
      • Tune Experiment Results (tune.ResultGrid)
      • Training in Tune (tune.Trainable, tune.report)
      • Tune Search Space API
      • Tune Search Algorithms (tune.search)
      • Tune Trial Schedulers (tune.schedulers)
      • Tune Stopping Mechanisms (tune.stopper)
      • Tune Console Output (Reporters)
      • Syncing in Tune
      • Tune Loggers (tune.logger)
      • Tune Callbacks (tune.Callback)
      • Environment variables used by Ray Tune
      • External library integrations for Ray Tune
      • Tune Internals
      • Tune CLI (Experimental)
    • Ray Serve
    • Ray RLlib
      • Algorithm Configuration API
        • AlgorithmConfig
        • build_algo
        • build_learner_group
        • build_learner
        • is_multi_agent
        • is_offline
        • learner_class
        • model_config
        • rl_module_spec
        • total_train_batch_size
        • get_default_learner_class
        • get_default_rl_module_spec
        • get_evaluation_config_object
        • get_multi_rl_module_spec
        • get_multi_agent_setup
        • get_rollout_fragment_length
        • copy
        • validate
        • freeze
      • Algorithms
        • Algorithm
        • setup
        • get_default_config
        • env_runner
        • eval_env_runner
        • train
        • training_step
        • save_to_path
        • restore_from_path
        • from_checkpoint
        • get_state
        • set_state
        • evaluate
        • get_module
        • add_policy
        • remove_policy
      • Callback APIs
        • RLlibCallback
        • on_algorithm_init
        • on_sample_end
        • on_train_result
        • on_evaluate_start
        • on_evaluate_end
        • on_env_runners_recreated
        • on_checkpoint_loaded
        • on_environment_created
        • on_episode_created
        • on_episode_start
        • on_episode_step
        • on_episode_end
      • Environments
        • EnvRunner API
        • SingleAgentEnvRunner API
        • SingleAgentEpisode API
        • MultiAgentEnv API
        • MultiAgentEnvRunner API
        • MultiAgentEpisode API
        • External Envs
        • Env Utils
      • RLModule APIs
        • RLModuleSpec
        • build
        • module_class
        • observation_space
        • action_space
        • inference_only
        • learner_only
        • model_config
        • MultiRLModuleSpec
        • build
        • DefaultModelConfig
        • RLModule
        • observation_space
        • action_space
        • inference_only
        • model_config
        • setup
        • as_multi_rl_module
        • forward_exploration
        • forward_inference
        • forward_train
        • _forward
        • _forward_exploration
        • _forward_inference
        • _forward_train
        • save_to_path
        • restore_from_path
        • from_checkpoint
        • get_state
        • set_state
        • MultiRLModule
        • setup
        • as_multi_rl_module
        • add_module
        • remove_module
        • save_to_path
        • restore_from_path
        • from_checkpoint
        • get_state
        • set_state
      • Distribution API
        • Distribution
        • from_logits
        • sample
        • rsample
        • logp
        • kl
      • LearnerGroup API
        • learners
        • build_learner_group
        • LearnerGroup
      • Offline RL API
        • offline_data
        • learners
        • env_runners
        • OfflineSingleAgentEnvRunner
        • OfflineData
        • __init__
        • sample
        • default_map_batches_kwargs
        • default_iter_batches_kwargs
        • OfflinePreLearner
        • __init__
        • SCHEMA
        • __call__
        • _map_to_episodes
        • _map_sample_batch_to_episode
        • _should_module_be_updated
      • ConnectorV2 API
      • Replay Buffer API
        • StorageUnit
        • ReplayBuffer
        • PrioritizedReplayBuffer
        • ReservoirReplayBuffer
        • sample
        • add
        • get_state
        • set_state
        • MultiAgentReplayBuffer
        • MultiAgentPrioritizedReplayBuffer
        • update_priorities_in_replay_buffer
        • sample_min_n_steps_from_buffer
      • RLlib Utilities
        • MetricsLogger
        • peek
        • log_value
        • log_dict
        • aggregate
        • log_time
        • Scheduler
        • validate
        • get_current_value
        • update
        • _create_tensor_variable
        • try_import_torch
        • clip_gradients
        • compute_global_norm
        • convert_to_torch_tensor
        • explained_variance
        • flatten_inputs_to_1d_tensor
        • global_norm
        • one_hot
        • reduce_mean_ignore_inf
        • sequence_mask
        • set_torch_seed
        • softmax_cross_entropy_with_logits
        • update_target_network
        • aligned_array
        • concat_aligned
        • convert_to_numpy
        • fc
        • flatten_inputs_to_1d_tensor
        • make_action_immutable
        • huber_loss
        • l2_loss
        • lstm
        • one_hot
        • relu
        • sigmoid
        • softmax
        • try_import_msgpack
        • Checkpointable
    • Ray Core
      • Core API
      • Scheduling API
      • Runtime Env API
      • Utility
      • Exceptions
      • Ray Core CLI
      • State CLI
      • State API
      • Ray Direct Transport (RDT) API
  • Ray Clusters
    • Key Concepts
    • Deploying on Kubernetes
      • Getting Started with KubeRay
        • KubeRay Operator Installation
        • RayCluster Quickstart
        • RayJob Quickstart
        • RayService Quickstart
        • RayCronJob Quickstart
      • User Guides
        • Deploy Ray Serve Apps
        • RayService worker Pods aren’t ready
        • RayService high availability
        • Enable High Throughput on Ray Serve with KubeRay
        • RayService Zero-Downtime Incremental Upgrades
        • KubeRay Observability
        • KubeRay upgrade guide
        • Managed Kubernetes services
        • Best Practices for Storage and Dependencies
        • RayCluster Configuration
        • KubeRay Autoscaling
        • KubeRay In-Place Pod Resizing (IPPR)
        • KubeRay label-based scheduling
        • GCS fault tolerance in KubeRay
        • Tuning Redis for a Persistent Fault Tolerant GCS
        • Configuring KubeRay to use Google Cloud Storage Buckets in GKE
        • Persist KubeRay custom resource logs
        • Persist KubeRay Operator Logs
        • Using GPUs
        • Use TPUs with KubeRay
        • Specify container commands for Ray head/worker Pods
        • Helm Chart RBAC
        • TLS Authentication
        • (Advanced) Understanding the Ray Autoscaler in the Context of Kubernetes
        • Use kubectl plugin (beta)
        • Configure Ray clusters to use token authentication
        • Configure Ray clusters to use Kubernetes RBAC authentication
        • Reducing image pull latency on Kubernetes
        • Using uv for Python package management in KubeRay
        • Use KubeRay dashboard (experimental)
        • Resource Isolation with Writable Cgroups on Google Kubernetes Engine (GKE)
        • Ray History Server with KubeRay
      • Examples
        • Train a PyTorch model on Fashion MNIST with CPUs on Kubernetes
        • Serve a StableDiffusion text-to-image model on Kubernetes
        • Serve a Stable Diffusion model on GKE with TPUs
        • Serve a MobileNet image classifier on Kubernetes
        • Serve a text summarizer on Kubernetes
        • RayJob Batch Inference Example
        • Priority Scheduling with RayJob and Kueue
        • Gang Scheduling with RayJob and Kueue
        • Distributed checkpointing with KubeRay and GCSFuse
        • Serve a Large Language Model using Ray Serve LLM on Kubernetes
        • Serve Deepseek R1 using Ray Serve LLM
        • Reinforcement Learning with Human Feedback (RLHF) for LLMs with verl on KubeRay
        • Deploying Ray Clusters via ArgoCD
        • Sandboxed Code Execution with Ray and Agent Sandbox
      • KubeRay Ecosystem
        • Ingress
        • KubeRay metrics references
        • Using Prometheus and Grafana
        • Profiling with py-spy
        • Gang scheduling, queue priority, and GPU sharing for RayClusters using KAI Scheduler
        • KubeRay integration with Volcano
        • KubeRay integration with Apache YuniKorn
        • Gang scheduling, Priority scheduling, and Autoscaling for KubeRay CRDs with Kueue
        • mTLS and L7 observability with Istio
        • KubeRay integration with scheduler plugins
      • KubeRay Benchmarks
        • KubeRay memory and scalability benchmark
      • KubeRay Troubleshooting
        • Troubleshooting guide
        • RayService troubleshooting
      • API Reference
    • Deploying on VMs
      • Getting Started
      • User Guides
        • Launching Ray Clusters on AWS, GCP, Azure, vSphere, On-Prem
        • Best practices for deploying large clusters
        • Configuring Autoscaling
        • Log Persistence
        • Community Supported Cluster Managers
      • Examples
        • Ray Train XGBoostTrainer on VMs
      • API References
        • Cluster Launcher Commands
        • Cluster YAML Configuration Options
    • Collecting and monitoring metrics
    • Configuring and Managing Ray Dashboard
    • Applications Guide
      • Ray Jobs Overview
        • Quickstart using the Ray Jobs CLI
        • Python SDK Overview
        • Python SDK API Reference
        • Ray Jobs CLI API Reference
        • Ray Jobs REST API
        • Ray Client
      • Programmatic Cluster Scaling
    • FAQ
    • Ray Cluster Management API
      • Cluster Management CLI
      • Python SDK API Reference
      • Ray Jobs CLI API Reference
      • Programmatic Cluster Scaling
    • Usage Stats Collection
  • Monitoring and Debugging
    • Ray Dashboard
    • Ray Distributed Debugger
    • Key Concepts
    • User Guides
      • Debugging Applications
        • Common Issues
        • Debugging Memory Issues
        • Debugging Hangs
        • Debugging Failures
        • Optimizing Performance
        • Ray Distributed Debugger
        • Using the Ray Debugger
      • Monitoring with the CLI or SDK
      • Configuring Logging
      • Profiling
      • Adding Application-Level Metrics
      • Tracing
      • Ray Event Export
    • Reference
      • State API
      • State CLI
      • System Metrics
  • Developer Guides
    • API stability
    • API policy
    • Getting involved and contributing
      • Building Ray from source
      • CI testing workflow on PRs
      • Contributing to the Ray documentation
      • How to write code snippets
      • Testing autoscaling locally
      • Tips for testing Ray programs
      • Debugging for Ray developers
      • Profiling for Ray developers
      • Using agents for development
    • Configuring Ray
    • Architecture whitepapers
  • Glossary
  • Security
    • Ray token authentication
  • Project Governance
    • People
  • Ray Clusters Overview
  • Application guide

Application guide#

This section introduces the main differences in running a Ray application on your laptop vs on a Ray Cluster. To get started, check out the job submissions page.

  • Ray Jobs Overview
    • Ray Jobs API
    • Running Jobs Interactively
    • Contents
  • Programmatic Cluster Scaling
    • ray.autoscaler.sdk.request_resources

previous

Configuring and Managing Ray Dashboard

next

Ray Jobs Overview
Edit on GitHub
Thanks for the feedback!
Was this helpful?
Yes
No
Feedback
Submit

© Copyright 2026, The Ray Team.

Created using Sphinx 8.2.3.

Built with the PyData Sphinx Theme 0.18.0.