--- title: How Databricks Uses AI to Accelerate Incident Investigation description: Databricks describes AI SRE, an internal AI-powered debugging agent that kicks off automatically when an incident fires, running parallel investigation tracks... canonical: https://daily.dev/posts/how-databricks-uses-ai-to-accelerate-incident-investigation-tyehhd3qg twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: How Databricks Uses AI to Accelerate Incident Investigation | daily.dev og:description: Databricks describes AI SRE, an internal AI-powered debugging agent that kicks off automatically when an incident fires, running parallel investigation tracks... og:url: https://daily.dev/posts/how-databricks-uses-ai-to-accelerate-incident-investigation-tyehhd3qg og:image: https://api.daily.dev/og/posts/TyEhhd3Qg.png og:image:alt: How Databricks Uses AI to Accelerate Incident Investigation og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # How Databricks Uses AI to Accelerate Incident Investigation **[databricks](https://daily.dev/sources/databricks)** · 9 min read · 5 upvotes · 2 comments ## Summary Databricks describes AI SRE, an internal AI-powered debugging agent that kicks off automatically when an incident fires, running parallel investigation tracks (platform health checks, service-level analysis, and runbook execution) to assemble diagnostic context before an on-call engineer even opens the alert. The system is built as a layered platform (primitives, API layer, core engine, application layer) so teams can encode their own runbooks as 'agentic runbooks' rather than relying on a single centralized agent. Design principles emphasize deterministic checks before open-ended LLM reasoning, transparency via traceable evidence, and graceful degradation when confidence is low. The tool now serves 150+ teams, 250+ weekly active users, and over 2,000 investigations daily, reportedly saving hours of debugging time and reducing MTTR. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How does Databricks use AI to speed up incident investigation across microservices? Databricks built an internal tool called AI SRE that automatically launches three parallel investigation tracks when an incident fires: platform health checks, service-level analysis of logs, metrics, and traces, and execution of team-specific runbooks. It assembles a diagnostic summary before the on-call engineer even opens the alert, and supports interactive natural-language follow-up questions. It now runs over 2,000 investigations daily across 150+ teams. _Teams designing similar AI-assisted debugging workflows can follow ongoing coverage of incident response tooling on daily.dev._ ### What engineering principles make an LLM-powered incident response agent trustworthy? Trust comes from running deterministic platform health checks and runbook steps before any open-ended LLM reasoning, so the LLM only synthesizes results rather than gathering data itself. Every conclusion links back to underlying evidence like a specific metric, log line, or deploy diff so engineers can verify rather than blindly trust it. The system also practices graceful degradation, admitting uncertainty rather than producing a hallucinated diagnosis. _Engineers evaluating AI reliability practices for production systems can track this kind of design guidance on daily.dev._ ## Community discussion Top comments from developers on daily.dev. **@trevorsuna** · 0 upvotes > Deterministic checks before open-ended reasoning make the investigation easier to trust and debug. The layered runbook model also lets service teams contribute domain knowledge without turning one central agent into an opaque collection of special cases. **@agustinbarrientos** · 0 upvotes > Every investigation should snapshot its evidence IDs and tool versions before the on-call engineer joins. That snapshot should remain available after resolution so later reviews can reproduce the starting point even when live telemetry has changed. ## Similar posts on daily.dev - [How we built an AI SRE agent that investigates like a team of engineers](https://daily.dev/posts/how-we-built-an-ai-sre-agent-that-investigates-like-a-team-of-engineers-w0ennf9ut) · Datadog · 0 upvotes · 0 comments - [AI SRE Done Right: Why Your Data Foundation Matters](https://daily.dev/posts/ai-sre-done-right-why-your-data-foundation-matters-qos3sgqu9) · Snowflake · 0 upvotes · 0 comments - [How AI Transformed Database Debugging at Databricks](https://daily.dev/posts/how-ai-transformed-database-debugging-at-databricks-xdhmgnzkw) · ByteByteGo · 8 upvotes · 0 comments - [AI DevOps vs. SRE agents: Compare AI incident response tools](https://daily.dev/posts/ai-devops-vs-sre-agents-compare-ai-incident-response-tools-mmosdnb8r) · The New Stack · 1 upvotes · 0 comments --- Tags: [#kubernetes](https://daily.dev/tags/kubernetes), [#ai-agents](https://daily.dev/tags/ai-agents), [#observability](https://daily.dev/tags/observability), [#sre](https://daily.dev/tags/sre) [View this post on daily.dev](https://daily.dev/posts/how-databricks-uses-ai-to-accelerate-incident-investigation-tyehhd3qg) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"How Databricks Uses AI to Accelerate Incident Investigation","url":"https://daily.dev/posts/how-databricks-uses-ai-to-accelerate-incident-investigation-tyehhd3qg","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/how-databricks-uses-ai-to-accelerate-incident-investigation-tyehhd3qg"},"datePublished":"2026-08-24T16:45:01.607Z","dateModified":"2026-09-13T18:45:11.166Z","description":"Databricks describes AI SRE, an internal AI-powered debugging agent that kicks off automatically when an incident fires, running parallel investigation tracks...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/7b297b0234d3bf5fcf823ee318afa977?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/7b297b0234d3bf5fcf823ee318afa977?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"databricks","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"databricks","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/fa7aa720f2db4d1eba826814730482c8","url":"https://daily.dev/sources/databricks"},"commentCount":2,"discussionUrl":"https://daily.dev/posts/how-databricks-uses-ai-to-accelerate-incident-investigation-tyehhd3qg","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":5},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":2}],"keywords":"kubernetes,ai-agents,observability,sre","timeRequired":"PT9M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"databricks","item":"https://daily.dev/sources/databricks"},{"@type":"ListItem","position":3,"name":"How Databricks Uses AI to Accelerate Incident Investigation"}]} {"@context":"https://schema.org","@type":"WebPage","@id":"https://daily.dev/posts/how-databricks-uses-ai-to-accelerate-incident-investigation-tyehhd3qg","comment":[{"@type":"Comment","text":"Deterministic checks before open-ended reasoning make the investigation easier to trust and debug. The layered runbook model also lets service teams contribute domain knowledge without turning one central agent into an opaque collection of special cases.","datePublished":"2026-08-25T14:57:40.975Z","url":"https://daily.dev/posts/TyEhhd3Qg#c-p8F8qpKgq","author":{"@type":"Person","name":"Trevor Suna","url":"https://daily.dev/trevorsuna","image":"https://media.daily.dev/image/upload/s--dZ7gXxpp--/f_auto/v1784081551/avatars/avatar_EMoP47rpuw8DNjhp6R1b6?_a=BAMAMicg0"}},{"@type":"Comment","text":"Every investigation should snapshot its evidence IDs and tool versions before the on-call engineer joins. That snapshot should remain available after resolution so later reviews can reproduce the starting point even when live telemetry has changed.","datePublished":"2026-08-26T07:31:08.927Z","url":"https://daily.dev/posts/TyEhhd3Qg#c-DrEMjJuhm","author":{"@type":"Person","name":"Agustin Barrientos","url":"https://daily.dev/agustinbarrientos","image":"https://media.daily.dev/image/upload/s--5ayxQnqn--/f_auto/v1788281802/avatars/avatar_wQYYVe5Tbj0NJ7C7qPoa8?_a=BAMAMicg0"}}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/how-databricks-uses-ai-to-accelerate-incident-investigation-tyehhd3qg#faq","mainEntity":[{"@type":"Question","name":"How does Databricks use AI to speed up incident investigation across microservices?","acceptedAnswer":{"@type":"Answer","text":"Databricks built an internal tool called AI SRE that automatically launches three parallel investigation tracks when an incident fires: platform health checks, service-level analysis of logs, metrics, and traces, and execution of team-specific runbooks. It assembles a diagnostic summary before the on-call engineer even opens the alert, and supports interactive natural-language follow-up questions. It now runs over 2,000 investigations daily across 150+ teams. Teams designing similar AI-assisted debugging workflows can follow ongoing coverage of incident response tooling on daily.dev."}},{"@type":"Question","name":"What engineering principles make an LLM-powered incident response agent trustworthy?","acceptedAnswer":{"@type":"Answer","text":"Trust comes from running deterministic platform health checks and runbook steps before any open-ended LLM reasoning, so the LLM only synthesizes results rather than gathering data itself. Every conclusion links back to underlying evidence like a specific metric, log line, or deploy diff so engineers can verify rather than blindly trust it. The system also practices graceful degradation, admitting uncertainty rather than producing a hallucinated diagnosis. Engineers evaluating AI reliability practices for production systems can track this kind of design guidance on daily.dev."}}]} ```