Ray Train Examples

Below are examples for using Ray Train with a variety of frameworks and use cases.

Beginner

Framework	Example
PyTorch	Train a Fashion MNIST Image Classifier with PyTorch
Lightning	Train an MNIST Image Classifier with Lightning
Transformers	Fine-tune a Text Classifier on the Yelp Reviews Dataset with Hugging Face Transformers
Accelerate	Distributed Data Parallel Training with Hugging Face Accelerate
DeepSpeed	Train with DeepSpeed ZeRO-3
TensorFlow	Train an MNIST Image Classifier with TensorFlow
Horovod	Train with Horovod and PyTorch

Intermediate

Framework	Example
PyTorch	Fine-tune of Stable Diffusion with DreamBooth and Ray Train
Lightning	Train with PyTorch Lightning and Ray Data
Transformers	Fine-tune a Text Classifier on GLUE Benchmark with Hugging Face Accelerate

Advanced

Framework	Example
Accelerate, DeepSpeed	Fine-tune Llama-2 series models with Deepspeed, Accelerate, and Ray Train TorchTrainer
Transformers, DeepSpeed	Fine-tune GPT-J-6B with Ray Train and DeepSpeed
Lightning, DeepSpeed	Fine-tune vicuna-13b with PyTorch Lightning and DeepSpeed
Lightning	Fine-tune dolly-v2-7b with PyTorch Lightning and FSDP