Building Your Own Jarvis: A Hands-on RAG Workshop

This hands-on workshop guides participants through creating a private AI assistant using Retrieval-Augmented Generation (RAG) with open-source LLMs in just one hour. The fast-paced session delivers practical, implementable knowledge for building a personalized "Jarvis" that can answer questions based on your own data.

Workshop Structure:

Setup & Fundamentals

• Quick introduction to RAG architecture and components

• Setting up a pre-configured Python environment with LangChain, embeddings, and Chroma DB

• Accessing a lightweight, pre-quantized open-source LLM (Llama 3 or Deepseek R1)

Knowledge Base Construction

• Document processing pipeline with pre-built helper functions

• Chunking strategies for optimal retrieval

• Creating and storing vector embeddings

• Hands-on activity: Participants process a small document collection

RAG Implementation

• Building the retrieval mechanism

• Crafting effective prompts for context integration

• Implementing a basic query-response loop

• Live coding: Connecting components into a working system

Testing & Next Steps

• Testing the assistant with sample questions

• Performance optimization tips

• Resources for further development

Participants will leave with a functioning prototype of their own RAG-powered assistant and the knowledge to extend it with their own data. Pre-built templates, LLMs and Colab notebooks will be provided to maximize hands-on time during this concentrated session.

Pre-requisites: LM Studio, ollama, anaconda, python, vscode is required on participants/attendees laptops

This workshop guides participants through building a personalized AI assistant using Retrieval-Augmented Generation (RAG) with open-source LLMs. Participants will:

• Deploy a local open-source LLM (Llama 3/Deepseek) for private inference

• Create an efficient document processing pipeline for knowledge ingestion

• Implement vector storage and retrieval mechanisms

• Master prompt engineering techniques for context-aware responses

• Optimize performance for resource-constrained environments

By session's end, attendees will have built their own "Jarvis" – a private AI assistant that leverages their personal or organizational knowledge. Ideal for developers and data scientists seeking to create customized AI tools while maintaining data privacy.

Prerequisites: Basic Python programming