--- title: Iris - Turning observations into actionable insights for... description: Iris is an observability platform for ETL jobs that captures critical job metrics, provides real-time monitoring and offline analytics, and offers actionable... canonical: https://daily.dev/posts/iris---turning-observations-into-actionable-insights-for-enhanced-decision-making-v6axp1oy9 twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Iris - Turning observations into actionable insights for enhanced decision making | daily.dev og:description: Iris is an observability platform for ETL jobs that captures critical job metrics, provides real-time monitoring and offline analytics, and offers actionable... og:url: https://daily.dev/posts/iris---turning-observations-into-actionable-insights-for-enhanced-decision-making-v6axp1oy9 og:image: https://api.daily.dev/og/posts/v6AXp1oy9.png og:image:alt: Iris - Turning observations into actionable insights for enhanced decision making og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Iris - Turning observations into actionable insights for enhanced decision making **[Grab Tech Blog](https://daily.dev/sources/grab)** · 17 min read · 2 upvotes · 1 comments ## Summary Iris is an observability platform for ETL jobs that captures critical job metrics, provides real-time monitoring and offline analytics, and offers actionable insights. It uses metrics collector, Kafka queue, and TIG stack for its platform architecture and collects data using JVM Profiler and sparkMeasure. Iris visualizes real-time data using Grafana and routes metrics to InfluxDB for offline analysis. It also employs K-means clustering for job classification and enhances accuracy in calculating Databricks infrastructure costs. Case studies demonstrate how Iris improves performance and cost efficiency. Future plans include developing APIs, adding a Presto listener, and creating a feedback loop for continuous improvement. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Community discussion Top comments from developers on daily.dev. **@jakob30** · 0 upvotes > Great post --- Tags: [#machine-learning](https://daily.dev/tags/machine-learning), [#data-science](https://daily.dev/tags/data-science), [#big-data](https://daily.dev/tags/big-data), [#kafka](https://daily.dev/tags/kafka), [#observability](https://daily.dev/tags/observability), [#spark](https://daily.dev/tags/spark), [#grafana](https://daily.dev/tags/grafana), [#influxdb](https://daily.dev/tags/influxdb), [#etl](https://daily.dev/tags/etl) [View this post on daily.dev](https://daily.dev/posts/iris---turning-observations-into-actionable-insights-for-enhanced-decision-making-v6axp1oy9) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Iris - Turning observations into actionable insights for enhanced decision making","url":"https://daily.dev/posts/iris---turning-observations-into-actionable-insights-for-enhanced-decision-making-v6axp1oy9","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/iris---turning-observations-into-actionable-insights-for-enhanced-decision-making-v6axp1oy9"},"datePublished":"2024-04-03T03:26:26.711Z","dateModified":"2024-05-09T08:43:33.910Z","description":"Iris is an observability platform for ETL jobs that captures critical job metrics, provides real-time monitoring and offline analytics, and offers actionable...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/dd2ea8bcf002b35b526f1958326c7168?_a=AQAEufR","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/dd2ea8bcf002b35b526f1958326c7168?_a=AQAEufR","isAccessibleForFree":true,"articleSection":"Grab Tech Blog","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Grab Tech Blog","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/c8b8850354ee4494a05777b62e42cc62","url":"https://daily.dev/sources/grab"},"commentCount":1,"discussionUrl":"https://daily.dev/posts/iris---turning-observations-into-actionable-insights-for-enhanced-decision-making-v6axp1oy9","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":2},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":1}],"keywords":"machine-learning,data-science,big-data,kafka,observability,spark,grafana,influxdb,etl","timeRequired":"PT17M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Grab Tech Blog","item":"https://daily.dev/sources/grab"},{"@type":"ListItem","position":3,"name":"Iris - Turning observations into actionable insights for enhanced decision making"}]} {"@context":"https://schema.org","@type":"WebPage","@id":"https://daily.dev/posts/iris---turning-observations-into-actionable-insights-for-enhanced-decision-making-v6axp1oy9","comment":[{"@type":"Comment","text":"Great post","datePublished":"2024-04-03T08:28:19.099Z","url":"https://daily.dev/posts/v6AXp1oy9#c-xuru9rl44","author":{"@type":"Person","name":"Jakob","url":"https://daily.dev/jakob30","image":"https://media.daily.dev/image/upload/s--O0TOmw4y--/f_auto/v1715772965/public/noProfile"}}]} ```