Ray Train: Scalable model training

Ray Train is a scalable machine learning library for distributed training and fine-tuning.

Use Ray Train to scale model training code from a single machine to a cluster of machines in the cloud, whether you have large models or large datasets. Ray Train abstracts away the complexities of distributed computing.

Ray Train supports many frameworks, including the following:

PyTorch ecosystem	More frameworks
PyTorch	TensorFlow
PyTorch Lightning	Keras
Hugging Face Transformers	Horovod
Hugging Face Accelerate	XGBoost
DeepSpeed	LightGBM

Install Ray Train

To install Ray Train, run the following command:

$ pip install -U "ray[train]"

To learn more about installing Ray and its libraries, see Installing Ray.

Get started

Overview

Understand the key concepts for distributed training with Ray Train.

Learn the basics

PyTorch

Get started with distributed model training using Ray Train and PyTorch.

Try Ray Train with PyTorch

PyTorch Lightning

Get started with distributed model training using Ray Train and Lightning.

Try Ray Train with Lightning

Hugging Face Transformers

Get started with distributed model training using Ray Train and Transformers.

Try Ray Train with Transformers

JAX

Get started with distributed model training using Ray Train and JAX.

Try Ray Train with JAX

Learn more

More frameworks

Don’t see your framework? See the guides for more frameworks.

Try Ray Train with other frameworks

User guides

Get how-to instructions for common training tasks with Ray Train.

Read how-to guides

Tutorials

Work through hands-on tutorials that cover ML workload patterns, from vision to recommendation systems.

Follow tutorials

Examples

Browse end-to-end code examples for different use cases.

Learn through examples

API

See the API reference for full descriptions of the Ray Train API.

Read the API reference