--- title: DREAM: Distributed RAG Experimentation Framework | daily.dev description: DREAM is a blueprint for distributed RAG experimentation using Ray, LlamaIndex, Ragas, MLFlow, and MinIO on Kubernetes. It helps in comparing and contrasting... canonical: https://daily.dev/posts/dream-distributed-rag-experimentation-framework-drrdetetv twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: DREAM: Distributed RAG Experimentation Framework | daily.dev og:description: DREAM is a blueprint for distributed RAG experimentation using Ray, LlamaIndex, Ragas, MLFlow, and MinIO on Kubernetes. It helps in comparing and contrasting... og:url: https://daily.dev/posts/dream-distributed-rag-experimentation-framework-drrdetetv og:image: https://api.daily.dev/og/posts/dRrDeteTV.png og:image:alt: DREAM: Distributed RAG Experimentation Framework og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # DREAM: Distributed RAG Experimentation Framework **[MLOps Community](https://daily.dev/sources/mlopscommunity)** · 7 min read · 1 upvotes · 1 comments ## Summary DREAM is a blueprint for distributed RAG experimentation using Ray, LlamaIndex, Ragas, MLFlow, and MinIO on Kubernetes. It helps in comparing and contrasting different combinations of RAG parameters to find the best one. The architecture includes technologies like Ray, LlamaIndex, ragas, MinIO, MLflow, Jupyter notebooks, Kubernetes, and ArgoCD. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Community discussion Top comments from developers on daily.dev. **@ahmetozel** · 0 upvotes > The thing that usually breaks these sweeps is combinatorics: chunk size x overlap x top-k x reranker is already hundreds of runs, and in my experience most of the variance comes from the chunking stage alone. Pinning the retriever and sweeping chunk boundaries first (semantic vs fixed, overlap ratio) narrows the grid a lot before you start burning GPU on generation configs. Also worth logging retrieval-only metrics separately from end-to-end answer quality - otherwise a good retriever hidden behind a weak prompt looks like a bad retriever. --- Tags: [#machine-learning](https://daily.dev/tags/machine-learning), [#llama](https://daily.dev/tags/llama) [View this post on daily.dev](https://daily.dev/posts/dream-distributed-rag-experimentation-framework-drrdetetv) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"DREAM: Distributed RAG Experimentation Framework","url":"https://daily.dev/posts/dream-distributed-rag-experimentation-framework-drrdetetv","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/dream-distributed-rag-experimentation-framework-drrdetetv"},"datePublished":"2024-05-15T16:14:13.400Z","dateModified":"2024-05-15T16:14:11.920Z","description":"DREAM is a blueprint for distributed RAG experimentation using Ray, LlamaIndex, Ragas, MLFlow, and MinIO on Kubernetes. It helps in comparing and contrasting...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/404e5e6649fac91283f7360a2045285e?_a=AQAEuiZ","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/404e5e6649fac91283f7360a2045285e?_a=AQAEuiZ","isAccessibleForFree":true,"articleSection":"MLOps Community","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"MLOps Community","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/85bb6afcf3f6475fb88db30f2bb53ece","url":"https://daily.dev/sources/mlopscommunity"},"commentCount":1,"discussionUrl":"https://daily.dev/posts/dream-distributed-rag-experimentation-framework-drrdetetv","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":1}],"keywords":"machine-learning,llama","timeRequired":"PT7M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"MLOps Community","item":"https://daily.dev/sources/mlopscommunity"},{"@type":"ListItem","position":3,"name":"DREAM: Distributed RAG Experimentation Framework"}]} {"@context":"https://schema.org","@type":"WebPage","@id":"https://daily.dev/posts/dream-distributed-rag-experimentation-framework-drrdetetv","comment":[{"@type":"Comment","text":"The thing that usually breaks these sweeps is combinatorics: chunk size x overlap x top-k x reranker is already hundreds of runs, and in my experience most of the variance comes from the chunking stage alone. Pinning the retriever and sweeping chunk boundaries first (semantic vs fixed, overlap ratio) narrows the grid a lot before you start burning GPU on generation configs. Also worth logging retrieval-only metrics separately from end-to-end answer quality - otherwise a good retriever hidden behind a weak prompt looks like a bad retriever.","datePublished":"2026-07-25T09:00:44.893Z","url":"https://daily.dev/posts/dRrDeteTV#c-u8QxYcwXa","author":{"@type":"Person","name":"Ahmet Özel","url":"https://daily.dev/ahmetozel","image":"https://avatars.githubusercontent.com/u/70992231?v=4"}}]} ```