Taming the Local LLM: Why MCP Doesn't Cure Hallucinations (A Dynamic Resume Case Study)

I wanted to replace my static PDF resume with an interactive AI agent. The architecture seemed straightforward: build a Python server using the Model Context Protocol (MCP), expose my structured SDET profile as dynamic tools, and connect a local Llama 3 model via Ollama to answer questions from recruiters.

I expected MCP to be the magic bridge that grounded the AI. Instead, the model confidently ignored my JSON profile. Despite my resume explicitly stating I have 4.5 years of experience in FinTech using Selenium and Python, the local LLM happily told users I had 8 years of experience in Healthcare building Cypress frameworks.

It didn't break the API contract. It didn't crash. It successfully called the tool, parsed the data, and then just lied.

The industry assumes that if you give an LLM a tool, the hallucination problem is solved. As an SDET, I discovered the reality is far messier. Building an MCP server is easy; forcing a stubborn, quantized local model to actually respect the constraints of that server is an entirely different engineering discipline.

This talk is an honest investigation into building the Dynamic Resume MCP Server. I will walk through the Python architecture, document exactly where the "Tool -> Grounding" pipeline breaks down, and share the mitigation strategies I tested to force local models to stick to the facts.

• Building the MCP Bridge: Learn how to translate structured data (like a professional profile) into discoverable Python MCP tools for local LLMs.

• The Anatomy of a Tool Hallucination: Explore the specific failure modes of local models, including Tool Skipping (answering directly instead of querying), Technology Swapping (inventing tools you don't use), and Context Ignoring (reading the JSON but prioritizing pre-trained weights).

• The Mitigation Experiments: Discover what actually happens when you try to fix these hallucinations. I will share the results of experimenting with stricter JSON schemas, modified tool descriptions, system prompt constraints, and temperature tuning.

• The Reality of AI Testing: Leave with an SDET’s perspective on the new testing paradigm: why traditional API testing fails for agentic workflows, and how to evaluate the reliability of non-deterministic reasoning paths.