Ray Train Examples
Contents
Ray Train Examples#
Below are examples for using Ray Train with a variety of frameworks and use cases.
Beginner#
Framework |
Example |
---|---|
PyTorch |
|
Lightning |
|
Transformers |
Fine-tune a Text Classifier on the Yelp Reviews Dataset with Hugging Face Transformers |
Accelerate |
Distributed Data Parallel Training with Hugging Face Accelerate |
DeepSpeed |
|
TensorFlow |
|
Horovod |
Intermediate#
Framework |
Example |
---|---|
PyTorch |
|
Lightning |
|
Transformers |
Fine-tune a Text Classifier on GLUE Benchmark with Hugging Face Accelerate |
Advanced#
Framework |
Example |
---|---|
Accelerate, DeepSpeed |
Fine-tune Llama-2 series models with Deepspeed, Accelerate, and Ray Train TorchTrainer |
Transformers, DeepSpeed |
|
Lightning, DeepSpeed |
|
Lightning |