Examples

Below is a list of tutorials that you can use to learn more about the different pieces of Ray Serve functionality and how to integrate different modeling frameworks.

Serve Examples

• Serving ML Models (Tensorflow, PyTorch, Scikit-Learn, others)
• Batching Tutorial
  • Define the Deployment
  • Deploy the Deployment
  • Deploy the Deployment using Python API
  • Troubleshooting
• Serving RLlib Models
• Scaling your Gradio app with Ray Serve
  • Dependencies
  • Quickstart: Deploy your Gradio app with Ray Serve
    • Deploying Gradio Server
  • Parallelizing models with Ray Serve
    • Original Approach
    • Parallelize using Ray Serve
• Visualizing a Deployment Graph with Gradio
  • Quickstart Example
    • Defining Nodes
    • Building the Graph
    • Deploy and Execute
  • Setting Up the Visualization
    • Requirement: Driver
    • Ensure Output Data is Properly Displayed
    • Providing Input
      • Single Input
      • Multiple Inputs
• Java Tutorial
  • Example Model
  • Converting to a Ray Serve Deployment
  • Testing the Ray Serve Deployment
  • Calling Ray Serve Deployment with HTTP
• Serving a Stable Diffusion Model
• Serving a Distilbert Model