From Prompts to Production: Building an Open-Source AI On-Call Agent with MCP and Multi-Agent Workflows

Large Language Models have made it incredibly easy to build AI demos, but taking an AI agent from a weekend prototype to a production-ready system is a completely different challenge.

In this talk, I’ll walk through the architecture and engineering decisions behind building an AI-powered on-call assistant using an open-source stack. Instead of focusing on prompt engineering alone, we’ll look at what happens after the first successful demo - how agents interact with tools, how Model Context Protocol (MCP) simplifies integrations, how multi-agent workflows improve task execution, and what it takes to deploy and operate these systems reliably.

We’ll explore an end-to-end architecture consisting of a web UI, agent layer, MCP server, tool integrations, and deployment on Kubernetes/OpenShift with GitLab CI/CD. Along the way, I’ll discuss real engineering challenges such as context management, tool failures, latency, deployment strategies, observability, and debugging production issues.

The goal of this talk isn’t to introduce another AI framework. Instead, it’s to share practical lessons that any engineer can apply while building production-grade AI applications using open technologies.

Whether you’re experimenting with AI agents for the first time or already deploying them in production, you’ll leave with a clearer understanding of the architectural patterns, trade-offs, and operational challenges involved in moving from prompts to production.

• Understand the architecture of a production-ready AI agent built using open-source components.

• Learn how the Model Context Protocol (MCP) simplifies tool integration and agent extensibility.

• Discover common production challenges including context management, latency, hallucinations, retries, and observability.

• See how GitLab CI/CD and Kubernetes/OpenShift can be used to automate deployments for AI services.

• Learn practical engineering patterns that make AI systems more reliable, maintainable, and scalable.

• Understand the trade-offs between proof-of-concept demos and production-ready AI applications.

• Leave with a reference architecture that can be adapted for your own AI projects.