User Guides

If you’re new to Ray Data, we recommend starting with the Ray Data Quick Start. This user guide will help you navigate the Ray Data project and show you how achieve several tasks.

• Loading Data
  • Generating Synthetic Data
  • Reading Files From Storage
  • From In-Memory Data
  • From Torch and TensorFlow
  • From 🤗 (Hugging Face) Datasets
  • From MongoDB
  • Reading From SQL Databases
  • Custom Datasources
  • Performance Considerations
• Transforming Data
  • Overview
  • Map transforms
  • Shuffle transforms
  • Grouped transforms
• Consuming Data
  • Retrieving a limited set of rows
  • Iterating over Datasets
  • Splitting Into and Consuming Shards
  • Saving Data
• Running Batch Inference
  • Why use Ray Data for batch inference?
  • Quick Start
  • Walk-through: Batch Inference with Ray
  • Advanced Guide to Batch Inference with PyTorch
  • Working with batch formats
  • Configuration & Troubleshooting
  • Using GPUs in batch inference
  • Learn more
• Working with Tensors
  • Tensor data representation
  • Transforming tensor data
  • Saving tensor data
• Scheduling, Execution, and Memory Management
  • Scheduling
  • Execution
  • Memory Management
• Performance Tips and Tuning
  • Monitoring your application
  • Debugging Statistics
  • Batching Transforms
  • Format Overheads
  • Parquet Column Pruning
  • Parquet Row Pruning
  • Tuning Read Parallelism
  • Tuning Read Resources
  • Enabling Push-Based Shuffle