Advanced topics

This section covers advanced Ray topics that go beyond the basic task, actor, and object APIs.

• Tips for first-time users
  • Tip 1: Delay ray.get()
  • Tip 2: Avoid tiny tasks
  • Tip 3: Avoid passing same object repeatedly to remote tasks
  • Tip 4: Pipeline data processing
• Type hints in Ray
  • Overview
  • Pattern 1: Use ray.remote as a function to build an actor
    • Why is this pattern necessary?
  • Pattern 2: Use @ray.method decorator for remote methods
• Starting Ray
  • What’s the Ray runtime?
  • Starting Ray on a single machine
  • Starting Ray through the CLI (ray start)
  • Launching a Ray cluster (ray up)
  • What’s next?
• Ray generators
  • Choosing a generator API
  • Getting started
  • Error handling
  • Generator from actor tasks
  • Using the Ray generator with asyncio
  • Garbage collection of object references
  • Fault tolerance
  • Cancellation
  • Wait for a generator without blocking a thread
    • Wait until a generator task completes
    • Use asyncio and await
    • Use ray.wait
  • Thread safety
  • Limitation
  • Deprecated dynamic generator
    • Dynamic generators
      • num_returns set by the task caller
      • num_returns set by the task executor
      • Exception handling
      • Limitations
• Use namespaces
  • Specify a namespace for a named actor
  • Anonymous namespaces
  • Get the current namespace
• Cross-language programming
  • Set up the driver
  • Call Java from Python
  • Call Python from Java
  • Cross-language data serialization
  • Cross-language exception stacks
• Use Ray with Jupyter Notebook and JupyterLab
  • Set up the notebook
• Lazy computation graphs with the Ray DAG API
  • Ray DAG with functions
  • Ray DAG with classes and class methods
  • Ray DAG with custom InputNode
  • Ray DAG with MultiOutputNode
  • Reuse Ray actors in DAGs
  • More resources
• Miscellaneous topics
  • Dynamic remote parameters
  • Overloaded functions
  • Inspecting cluster state
    • Node information
    • Resource information
  • Running large Ray clusters
    • Tuning operating system settings
      • Maximum open files
      • ARP cache
    • Benchmark
• Authenticate remote URIs in runtime_env
  • Keep credentials out of remote URIs
  • Use a netrc file on VMs
  • Use a netrc secret on KubeRay
  • Use bearer tokens for HTTPS authentication
    • Use a bearer token secret on KubeRay
• Lifetimes of user-spawned processes
  • User-spawned process killed on worker exit
  • Enable the subreaper feature
  • Caution: The core worker reaps zombie processes
  • How does the subreaper work?
• Head node memory management
  • Why head node memory grows
  • Mitigation strategies
    • Avoid scheduling on the head node
    • Disable the dashboard
  • Kubernetes configuration
    • Head pod memory settings
    • Recommended head node specifications
  • Best practices
  • Troubleshooting