End-to-end RAG Deep Dive

This tutorial covers end-to-end Retrieval-Augmented Generation (RAG) pipelines using Ray, from data ingestion and LLM deployment to prompt engineering, evaluation and scaling out all workloads in the application.

• End-to-end RAG Deep Dive
  • Notebooks
• Build a Regular RAG Document Ingestion Pipeline (No Ray required)
  • Introduction
  • Document Loading with Unstructured IO
  • Chunking Strategies
  • Test the Chunking Strategy Implementation
  • Generate Embeddings
  • Set up Chroma DB
  • Embedding Search with Chroma
  • Reformat Chroma Search Results
  • Why it’s not scalable
• Scalable RAG Data Ingestion and Pagination with Ray Data
  • Loading Documents from S3
  • Scalable Document Processing and Pagination
  • Text Processing Pipeline
  • Verifying and Searching the Data
  • Observations
  • Summary
• Deploy LLM with Ray Serve LLM
  • Dependancies
  • Setting Up Your LLM Deployment
  • Streaming Chat Completions with OpenAI Client
  • Shut down the service
  • Production Deployment
  • Building an LLM Client for Chat Interactions
  • Query the Service
• Build Basic RAG App
  • Prerequisites
  • Verify the LLM service
  • Observation and Why we need RAG?
  • Build the embedder
  • Query the Chroma DB
  • Render the basic RAG prompt
  • Connect the Components of RAG
  • Get the Final Response
  • Observations and Next Steps
• Improve RAG with Prompt Engineering
  • Prerequisites
  • Initlize the RAG components
  • Basic RAG Prompt
  • Now let’s Upgrade to an Advanced Prompt
  • Put the New Prompt in Action
  • Add Chat History for RAG
  • Generate Citation Tokens in RAG response
  • Replace Citation Tokens with Actual Links
  • Obeservations
• Evaluate RAG with Online Inference
  • Prerequisites
  • Initlize the RAG components
  • Load the Evaluation Data
  • Evaluate the RAG Pipeline Using Online Inference
  • Evaluate the Results and Improve RAG Quality
  • Scalability Considerations: Why Online Inference May Not Be Ideal
• Evaluate RAG using Batch Inference with Ray Data LLM
  • How to Decide Between Online vs. Offline Inference for LLM
  • Key Benefits of Using Batch Inference with Ray Data LLM
  • Prerequisites
  • Load the Evaluation Data
  • Generating Embeddings from User Requests
  • Querying the Vector Store and Generating Prompts
  • Configuring and Running LLM Inference
  • Saving the Batch Inference Results
  • Visualize the Results
  • Evaluate Results and Imrpove RAG Quality
  • Final Notes