Examples
Examples#
Below is a list of tutorials that you can use to learn more about the different pieces of Ray Serve functionality and how to integrate different modeling frameworks.
Serve Examples
- Serving ML Models (Tensorflow, PyTorch, Scikit-Learn, others)
- Serving a Stable Diffusion Model
- Serving a Distilbert Model
- Serving an Object Detection Model
- Serving an inference model on AWS NeuronCores using FastAPI (Experimental)
- Serving RLlib Models
- Scaling your Gradio app with Ray Serve
- Batching Tutorial
- Streaming Tutorial
- Create a Streaming Deployment
- Stream inputs and outputs using WebSockets
- Batch requests and stream the output for each
- Visualizing a Deployment Graph with Gradio
- Java Tutorial