Datadog's security engineering team shares how they built MCP tools for Cloud SIEM, covering three core challenges: scoping tools around real user behavior (using API call patterns and RUM data), managing a shared context window across a growing multi-team toolset via progressive disclosure and query-based triage, and testing non-deterministic agent behavior with a custom eval framework integrated into CI. Key findings include that 44% of user messages involved detection rule authoring, 25% of customers triaged signals in bulk, and redesigning the bulk triage tool to accept search queries instead of signal ID lists reduced triage time from 2+ minutes for 50 signals to ~1 minute for hundreds. A lightweight governance committee enforces minimum eval scores before tools reach customers without slowing iteration.
