--- title: Building a hybrid demand forecasting service: classical... description: A design blueprint for a hybrid retail demand forecasting service that combines classical ML (XGBoost/LightGBM) for numeric predictions with an LLM for... canonical: https://daily.dev/posts/building-a-hybrid-demand-forecasting-service-classical-ml-for-the-number-an-llm-for-the-why-bfbys6aak twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Building a hybrid demand forecasting service: classical ML for the number, an LLM for the why | daily.dev og:description: A design blueprint for a hybrid retail demand forecasting service that combines classical ML (XGBoost/LightGBM) for numeric predictions with an LLM for... og:url: https://daily.dev/posts/building-a-hybrid-demand-forecasting-service-classical-ml-for-the-number-an-llm-for-the-why-bfbys6aak og:image: https://api.daily.dev/og/posts/bfBYS6aAk.png og:image:alt: Building a hybrid demand forecasting service: classical ML for the number, an LLM for the why og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Building a hybrid demand forecasting service: classical ML for the number, an LLM for the why **[Medium](https://daily.dev/sources/medium_js)** · 17 min read · 1 upvotes · 0 comments ## Summary A design blueprint for a hybrid retail demand forecasting service that combines classical ML (XGBoost/LightGBM) for numeric predictions with an LLM for generating human-readable explanations. The post covers two key pitfalls: data leakage from improper train/test splits in time series, and the lack of datasets that combine demand series with rich text. It proposes two data strategies (deriving series from Amazon reviews, or joining Favorita demand data with holiday event descriptions), walks through feature engineering with proper lag/rolling-window construction, explains walk-forward backtesting with TimeSeriesSplit and MASE evaluation, and details two LLM roles: feature generator (structured sentiment/aspect extraction) and narrator (grounded explanation). Evaluation approaches for both sides are covered, including Azure AI Foundry's Groundedness evaluator. The architecture targets Microsoft Fabric/OneLake for data, Azure ML managed endpoints for the classical model, and Foundry Agent Service for the LLM layer. The author is transparent that this is a design blueprint, not a deployed system. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How do I avoid data leakage when building rolling window features for time series forecasting? Shift the target series back one step with shift(1) before computing rolling means or standard deviations, so today's value never leaks into its own feature. A rolling(7).mean() computed directly on the target includes the current day's value, which the model would not have available at prediction time, producing an unrealistically optimistic backtest. _Developers building leak-free forecasting pipelines can track feature engineering pitfalls like this via daily.dev._ ### Why does a random train/test split give a misleading accuracy score for time series forecasting models? A random split puts future observations into training and past observations into test, letting the model effectively see the future before being evaluated, which makes the error estimate fantasy. Time-ordered validation, such as scikit-learn's TimeSeriesSplit with expanding windows and a gap parameter, avoids this by keeping training data strictly earlier than test data. _Anyone validating forecasting models turns to daily.dev to avoid backtesting mistakes like this._ ### What is MASE and why is it recommended over MAPE for evaluating demand forecasts? MASE (Mean Absolute Scaled Error) scales the forecast error by the in-sample MAE of a seasonal-naive baseline, so a value of 1 means the model performs as well as naive, below 1 beats it, and above 1 is worse. Unlike MAPE, which is undefined at zero and unstable near it, MASE is scale-free, comparable across series with different volumes, and works with intermittent or near-zero retail demand. Hyndman and Koehler recommend it as the best general-purpose accuracy measure. _Teams choosing forecast accuracy metrics can keep up with practices like MASE through daily.dev._ ## Similar posts on daily.dev - [2 Years of ML vs. 1 Month of Prompting](https://daily.dev/posts/2-years-of-ml-vs-1-month-of-prompting-jmutxz5qr) · Hacker News · 2 upvotes · 1 comments - [Getting Back to Basics](https://daily.dev/posts/getting-back-to-basics-8dse6eor7) · Simple Thread · 75 upvotes · 2 comments - [Inside Target’s LLM-Based System for Semantic Matching in Marketing Forecast Pipelines](https://daily.dev/posts/inside-target-s-llm-based-system-for-semantic-matching-in-marketing-forecast-pipelines-qkwajf1gd) · InfoQ · 2 upvotes · 1 comments - [Our Early Journey to Transform Instacart’s Discovery Recommendations with LLMs](https://daily.dev/posts/our-early-journey-to-transform-instacart-s-discovery-recommendations-with-llms-wpv1ryxzx) · Instacart · 1 upvotes · 0 comments - [LLM-Powered Time-Series Analysis](https://daily.dev/posts/llm-powered-time-series-analysis-jdiffr0zs) · Towards Data Science · 2 upvotes · 0 comments --- Tags: [#machine-learning](https://daily.dev/tags/machine-learning), [#python](https://daily.dev/tags/python), [#data-science](https://daily.dev/tags/data-science), [#llm](https://daily.dev/tags/llm) [View this post on daily.dev](https://daily.dev/posts/building-a-hybrid-demand-forecasting-service-classical-ml-for-the-number-an-llm-for-the-why-bfbys6aak) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Building a hybrid demand forecasting service: classical ML for the number, an LLM for the why","url":"https://daily.dev/posts/building-a-hybrid-demand-forecasting-service-classical-ml-for-the-number-an-llm-for-the-why-bfbys6aak","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/building-a-hybrid-demand-forecasting-service-classical-ml-for-the-number-an-llm-for-the-why-bfbys6aak"},"datePublished":"2026-07-12T13:51:46.576Z","dateModified":"2026-09-14T08:27:39.980Z","description":"A design blueprint for a hybrid retail demand forecasting service that combines classical ML (XGBoost/LightGBM) for numeric predictions with an LLM for...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/cbfc1030572ce2109becc6d7058296cd?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/cbfc1030572ce2109becc6d7058296cd?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Medium","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Medium","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/medium","url":"https://daily.dev/sources/medium_js"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/building-a-hybrid-demand-forecasting-service-classical-ml-for-the-number-an-llm-for-the-why-bfbys6aak","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"machine-learning,python,data-science,llm","timeRequired":"PT17M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Medium","item":"https://daily.dev/sources/medium_js"},{"@type":"ListItem","position":3,"name":"Building a hybrid demand forecasting service: classical ML for the number, an LLM for the why"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/building-a-hybrid-demand-forecasting-service-classical-ml-for-the-number-an-llm-for-the-why-bfbys6aak#faq","mainEntity":[{"@type":"Question","name":"How do I avoid data leakage when building rolling window features for time series forecasting?","acceptedAnswer":{"@type":"Answer","text":"Shift the target series back one step with shift(1) before computing rolling means or standard deviations, so today's value never leaks into its own feature. A rolling(7).mean() computed directly on the target includes the current day's value, which the model would not have available at prediction time, producing an unrealistically optimistic backtest. Developers building leak-free forecasting pipelines can track feature engineering pitfalls like this via daily.dev."}},{"@type":"Question","name":"Why does a random train/test split give a misleading accuracy score for time series forecasting models?","acceptedAnswer":{"@type":"Answer","text":"A random split puts future observations into training and past observations into test, letting the model effectively see the future before being evaluated, which makes the error estimate fantasy. Time-ordered validation, such as scikit-learn's TimeSeriesSplit with expanding windows and a gap parameter, avoids this by keeping training data strictly earlier than test data. Anyone validating forecasting models turns to daily.dev to avoid backtesting mistakes like this."}},{"@type":"Question","name":"What is MASE and why is it recommended over MAPE for evaluating demand forecasts?","acceptedAnswer":{"@type":"Answer","text":"MASE (Mean Absolute Scaled Error) scales the forecast error by the in-sample MAE of a seasonal-naive baseline, so a value of 1 means the model performs as well as naive, below 1 beats it, and above 1 is worse. Unlike MAPE, which is undefined at zero and unstable near it, MASE is scale-free, comparable across series with different volumes, and works with intermittent or near-zero retail demand. Hyndman and Koehler recommend it as the best general-purpose accuracy measure. Teams choosing forecast accuracy metrics can keep up with practices like MASE through daily.dev."}}]} ```