Instacart built Blueberry, a Slack-native AI reasoning harness for on-call engineers that auto-triages alerts and supports multi-turn investigations. The system uses a Postgres-backed durable job queue, three composable Model Context Protocol surfaces (in-process, shared, and team-hosted), and parallel sub-agents to gather historical and live evidence simultaneously. In April 2026, it ran ~25k diagnostic passes across 270+ Slack channels, achieving ~3-minute average time to first insight and ~3-minute theory-testing turnaround with a 99.9% success rate. Key design lessons: harness quality (grounding, specialization, continuity, learning loops) matters more than model quality alone. Teams can contribute custom profiles, runbooks, and MCP endpoints to make tribal knowledge reusable infrastructure.

15m read timeFrom tech.instacart.com
Post cover image
Table of contents
On-call is mostly a clarity and speed problemBlueberry is a Slack-native on-call reasoning harnessArchitecture: a durable, tool-aware reasoning harnessReasoning ladder: where the agent gets freedom, where we hold the lineFrom alert to first insightExample: SEV2 before declarationFrom first insight to tested theoriesExample: 8-turn PR exoneration ending at a smoking-gun WARN logGet Karthik Halukurike ’s stories in your inbox
68 Impressions