Observability

Observability, Debugging, and Profiling

• Overview
  • Application Logging
  • Exceptions
  • Accessing Ray States
  • Dashboard (Web UI)
  • Debugger
  • Monitoring Cluster State and Resource Demands
  • Metrics
  • Profiling
  • Tracing
• Ray Dashboard
  • Getting Started
  • Views
  • Advanced Usage
  • Viewing built-in dashboard API metrics
  • References
• Monitoring Ray States
  • Getting Started
  • Key Concepts
  • Summary
  • List
  • Get
  • Logs
  • Failure Semantics
  • API Reference
• Ray Debugger
  • Getting Started
  • Running on a Cluster
  • Debugger Commands
  • Stepping between Ray tasks
  • Post Mortem Debugging
  • Debugging APIs
• Logging
  • Driver logs
  • Job logs
  • Worker logs
  • Disabling logging to the driver
  • Customizing Actor logs prefixes
  • How to set up loggers
  • How to use structured logging
  • Redirecting Ray logs to stderr
• Metrics
  • Getting Started
  • Grafana
  • System Metrics
  • Application-level Metrics
  • Customize prometheus export port
  • Troubleshooting
• Tracing
  • Getting Started
  • Tracing Startup Hook
• Debugging (internal)
  • Starting processes in a debugger
  • Backend logging
  • Backend event stats
  • Callback latency injection
• Profiling (internal)
  • Getting a stack trace of Ray C++ processes
  • Installation
  • Launching the to-profile binary
  • Memory Profiling
  • Visualizing the CPU profile
  • Running Microbenchmarks
  • References