--- title: "How Databricks Uses AI to Accelerate Incident Investigation" url: https://daily.dev/posts/how-databricks-uses-ai-to-accelerate-incident-investigation-tyehhd3qg source_url: https://www.databricks.com/blog/how-databricks-uses-ai-accelerate-incident-investigation type: article source: "databricks" published: 2026-08-24T16:45:01.607Z updated: 2026-08-24T16:45:27.989Z tags: ["kubernetes", "ai-agents", "observability", "sre"] reading_time: 9 upvotes: 5 comments: 2 language: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # How Databricks Uses AI to Accelerate Incident Investigation **[databricks](https://daily.dev/sources/databricks)** · 9 min read · 5 upvotes · 2 comments ## Summary Databricks describes AI SRE, an internal AI-powered debugging agent that kicks off automatically when an incident fires, running parallel investigation tracks (platform health checks, service-level analysis, and runbook execution) to assemble diagnostic context before an on-call engineer even opens the alert. The system is built as a layered platform (primitives, API layer, core engine, application layer) so teams can encode their own runbooks as 'agentic runbooks' rather than relying on a single centralized agent. Design principles emphasize deterministic checks before open-ended LLM reasoning, transparency via traceable evidence, and graceful degradation when confidence is low. The tool now serves 150+ teams, 250+ weekly active users, and over 2,000 investigations daily, reportedly saving hours of debugging time and reducing MTTR. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What is Databricks' AI SRE system and how does it help with incident investigation? AI SRE is an internal AI-powered debugging agent at Databricks that automatically begins investigating as soon as an incident fires, running three parallel tracks: platform health checks, service-level log/metric/trace analysis, and execution of team-specific agentic runbooks. It assembles a diagnostic summary before the on-call engineer even opens the alert, and supports over 150 teams with 250+ weekly active users running 2,000+ investigations daily. _daily.dev surfaces engineering deep dives like this for teams designing AI-assisted incident response workflows._ ### How is Databricks' AI SRE platform architected into layers? It uses four layers: primitives (raw operational data like metrics, logs, alerts, deployments), an API layer providing uniform authenticated access via purpose-specific APIs like Observability, Deployment, and Alerts APIs, a core engine handling orchestration and LLM-powered synthesis via a bot framework, and an application layer where the triage bot and third-party AI tools run. This separation lets data access and orchestration evolve independently of debugging workflows. _engineers designing layered observability platforms can follow architecture writeups like this on daily.dev._ ### Why does Databricks run deterministic health checks before letting an LLM reason about an incident? Because trust is critical during incident response, Databricks runs deterministic platform health checks and runbook steps first, and only lets the LLM synthesize and explain results rather than gather data on its own judgment. Every conclusion links back to underlying evidence like specific metrics, log lines, and deploy diffs so engineers can audit rather than blindly trust recommendations, and the system explicitly reports when it cannot determine a root cause with confidence. _teams weighing how much autonomy to give AI agents in production can track these design tradeoffs on daily.dev._ ## Community discussion Top comments from developers on daily.dev. **@trevorsuna** · 0 upvotes > Deterministic checks before open-ended reasoning make the investigation easier to trust and debug. The layered runbook model also lets service teams contribute domain knowledge without turning one central agent into an opaque collection of special cases. **@agustinbarrientos** · 0 upvotes > Every investigation should snapshot its evidence IDs and tool versions before the on-call engineer joins. That snapshot should remain available after resolution so later reviews can reproduce the starting point even when live telemetry has changed. ## Similar posts on daily.dev - [How we built an AI SRE agent that investigates like a team of engineers](https://daily.dev/posts/how-we-built-an-ai-sre-agent-that-investigates-like-a-team-of-engineers-w0ennf9ut) · Datadog · 0 upvotes · 0 comments - [AI SRE Done Right: Why Your Data Foundation Matters](https://daily.dev/posts/ai-sre-done-right-why-your-data-foundation-matters-qos3sgqu9) · Snowflake · 0 upvotes · 0 comments --- Tags: [#kubernetes](https://daily.dev/tags/kubernetes), [#ai-agents](https://daily.dev/tags/ai-agents), [#observability](https://daily.dev/tags/observability), [#sre](https://daily.dev/tags/sre) [View this post on daily.dev](https://daily.dev/posts/how-databricks-uses-ai-to-accelerate-incident-investigation-tyehhd3qg)