A deep-dive into the Data Governance Copilot architecture (Part 3), covering container-level design with SvelteKit frontend, FastAPI backend, and an agentic loop integrating OpenShift AI with the PG Airman MCP server. Explains two deployment modes: Red Hat Integrated Llama Stack versus lower-level MCP-direct using the OpenAI SDK and MCP Python library. Details how inbound LLM messages use OpenAI API format converted via Jinja tokenizer templates in vLLM, and how outbound tool calls are parsed — including a custom client-side parser needed for NVIDIA Nemotron Nano 9B due to its lack of streaming support in the native vLLM plugin. Compares Nemotron's XML-tagged TOOLCALL format against Qwen3's standard hermes parser output.
Table of contents
Behind the curtain: The copilot architectureLlama Stack vs. lower-level Python librariesUser query flow and a simple agentic loopInbound and outbound LLM messagesCustom tool call parsingNext steps275 Impressions