.. _tune-examples-ref: .. _tune-recipes: ================= Ray Tune Examples ================= .. tip:: See :ref:`overview` to learn more about Tune features. Below are examples for using Ray Tune for a variety use cases and sorted by categories: * `ML frameworks`_ * `Experiment tracking tools`_ * `Hyperparameter optimization frameworks`_ * `Others`_ * `Exercises`_ .. _ml-frameworks: ML frameworks ------------- .. toctree:: :hidden: PyTorch Example PyTorch Lightning Example XGBoost Example LightGBM Example Hugging Face Transformers Example Ray RLlib Example Keras Example Horovod Example Ray Tune integrates with many popular machine learning frameworks. Here you find a few practical examples showing you how to tune your models. At the end of these guides you will often find links to even more examples. .. list-table:: * - :doc:`How to use Tune with Keras and TensorFlow models ` * - :doc:`How to use Tune with PyTorch models ` * - :doc:`How to tune PyTorch Lightning models ` * - :doc:`Tuning RL experiments with Ray Tune and Ray Serve ` * - :doc:`Tuning XGBoost parameters with Tune ` * - :doc:`Tuning LightGBM parameters with Tune ` * - :doc:`Tuning Horovod parameters with Tune ` * - :doc:`Tuning Hugging Face Transformers with Tune ` * - :doc:`End-to-end example for tuning a TensorFlow model <../../train/examples/tf/tune_tensorflow_mnist_example>` * - :doc:`End-to-end example for tuning a PyTorch model with PBT <../../train/examples/pytorch/tune_cifar_torch_pbt_example>` .. _experiment-tracking-tools: Experiment tracking tools ------------------------- .. toctree:: :hidden: Weights & Biases Example MLflow Example Aim Example Comet Example Ray Tune integrates with some popular Experiment tracking and management tools, such as CometML, or Weights & Biases. For how to use Ray Tune with Tensorboard, see :ref:`Guide to logging and outputs `. .. list-table:: * - :doc:`Using Aim with Ray Tune for experiment management ` * - :doc:`Using Comet with Ray Tune for experiment management ` * - :doc:`Tracking your experiment process Weights & Biases ` * - :doc:`Using MLflow tracking and auto logging with Tune ` .. _hyperparameter-optimization-frameworks: Hyperparameter optimization frameworks -------------------------------------- .. toctree:: :hidden: Ax Example HyperOpt Example Bayesopt Example BOHB Example Nevergrad Example Optuna Example Tune integrates with a wide variety of hyperparameter optimization frameworks and their respective search algorithms. See the following detailed examples for each integration: .. list-table:: * - :doc:`ax_example` * - :doc:`hyperopt_example` * - :doc:`bayesopt_example` * - :doc:`bohb_example` * - :doc:`nevergrad_example` * - :doc:`optuna_example` .. _tune-examples-others: Others ------ .. list-table:: * - :doc:`Simple example for doing a basic random and grid search ` * - :doc:`Example of using a simple tuning function with AsyncHyperBandScheduler ` * - :doc:`Example of using a trainable function with HyperBandScheduler and the AsyncHyperBandScheduler ` * - :doc:`Configuring and running (synchronous) PBT and understanding the underlying algorithm behavior with a simple example ` * - :doc:`includes/pbt_function` * - :doc:`includes/pb2_example` * - :doc:`includes/logging_example` .. _tune-examples-exercises: Exercises --------- Learn how to use Tune in your browser with the following Colab-based exercises. .. list-table:: :widths: 50 30 20 :header-rows: 1 * - Description - Library - Colab link * - Basics of using Tune - Pytorch - .. image:: https://colab.research.google.com/assets/colab-badge.svg :target: https://colab.research.google.com/github/ray-project/tutorial/blob/master/tune_exercises/exercise_1_basics.ipynb :alt: Open in Colab * - Using search algorithms and trial schedulers to optimize your model - Pytorch - .. image:: https://colab.research.google.com/assets/colab-badge.svg :target: https://colab.research.google.com/github/ray-project/tutorial/blob/master/tune_exercises/exercise_2_optimize.ipynb :alt: Open in Colab * - Using Population-Based Training (PBT) - Pytorch - .. image:: https://colab.research.google.com/assets/colab-badge.svg :target: https://colab.research.google.com/github/ray-project/tutorial/blob/master/tune_exercises/exercise_3_pbt.ipynb" target="_parent :alt: Open in Colab * - Fine-tuning Hugging Face Transformers with PBT - Hugging Face Transformers and Pytorch - .. image:: https://colab.research.google.com/assets/colab-badge.svg :target: https://colab.research.google.com/drive/1tQgAKgcKQzheoh503OzhS4N9NtfFgmjF?usp=sharing :alt: Open in Colab * - Logging Tune runs to Comet ML - Comet - .. image:: https://colab.research.google.com/assets/colab-badge.svg :target: https://colab.research.google.com/drive/1dp3VwVoAH1acn_kG7RuT62mICnOqxU1z?usp=sharing :alt: Open in Colab Tutorial source files are on `GitHub `_.