Building a Multifaceted Agentic AI Evaluation Framework for Helix

Two Six Technologies describes building an evaluation framework for Helix, its agentic AI orchestrator used in national security settings. The framework combines a trace-and-check test harness (originally built for a system called Sentr) with a domain-specific evaluation suite that benchmarks LLMs, workflows, and prompt strategies. Rather than simple pass/fail checks, it tracks latency, token usage, cost, citation quality, and tool-call behavior over time to detect performance drift. Future plans include integrating these checks into CI/CD pipelines and adding live monitoring via Langfuse and Kibana to catch degradation before it affects users.