--- title: Product Experimentation with Instrumental Variables:... description: Standard regression is fundamentally biased when measuring LLM routing quality because routing rules correlate with query complexity, a confounder that... canonical: https://daily.dev/posts/product-experimentation-with-instrumental-variables-unconfounding-llm-routing-decisions-in-python-dxangqznt twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Product Experimentation with Instrumental Variables: Unconfounding LLM Routing Decisions in Python | daily.dev og:description: Standard regression is fundamentally biased when measuring LLM routing quality because routing rules correlate with query complexity, a confounder that... og:url: https://daily.dev/posts/product-experimentation-with-instrumental-variables-unconfounding-llm-routing-decisions-in-python-dxangqznt og:image: https://api.daily.dev/og/posts/DxaNgQZnt.png og:image:alt: Product Experimentation with Instrumental Variables: Unconfounding LLM Routing Decisions in Python og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Product Experimentation with Instrumental Variables: Unconfounding LLM Routing Decisions in Python **[freeCodeCamp](https://daily.dev/sources/freecodecamp)** · 21 min read · 2 upvotes · 0 comments ## Summary Standard regression is fundamentally biased when measuring LLM routing quality because routing rules correlate with query complexity, a confounder that distorts performance estimates. Using a synthetic 50,000-query dataset, this tutorial demonstrates how to apply Two-Stage Least Squares (2SLS) with rate-limit fallbacks as an instrumental variable to recover an unbiased causal estimate. The walkthrough covers: why OLS underestimates the true premium routing effect (3.3 pp vs. 6 pp ground truth), building 2SLS from scratch in Python with statsmodels, validating instrument strength via the first-stage F-statistic, understanding the Local Average Treatment Effect (LATE) vs. ATE distinction, and computing bootstrap confidence intervals to correct for underestimated standard errors in manual 2SLS. The article also covers failure modes including weak instruments, exclusion restriction violations, and monotonicity assumptions, and recommends linearmodels.IV2SLS for production-grade analyses. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [Product Experimentation with Regression-Based Causal Inference: Estimating LLM Feature Impact with Python and statsmodels](https://daily.dev/posts/product-experimentation-with-regression-based-causal-inference-estimating-llm-feature-impact-with-p-cgaks9y2k) · freeCodeCamp · 0 upvotes · 0 comments - [Product Experimentation with Regression Discontinuity: How an LLM Confidence Threshold Creates a Natural Experiment in Python](https://daily.dev/posts/product-experimentation-with-regression-discontinuity-how-an-llm-confidence-threshold-creates-a-nat-gpkwxoqva) · freeCodeCamp · 1 upvotes · 0 comments - [Product Experimentation with Propensity Scores: Causal Inference for LLM-Based Features in Python](https://daily.dev/posts/product-experimentation-with-propensity-scores-causal-inference-for-llm-based-features-in-python-uju401omp) · freeCodeCamp · 4 upvotes · 0 comments - [Product Experimentation with Synthetic Control: Causal Inference for Global LLM Rollouts in Python](https://daily.dev/posts/product-experimentation-with-synthetic-control-causal-inference-for-global-llm-rollouts-in-python-ndk7jj8dt) · freeCodeCamp · 1 upvotes · 0 comments - [Beyond the Straight Line: Choosing Between OLS, Interaction Terms, and Tweedie Regression](https://daily.dev/posts/beyond-the-straight-line-choosing-between-ols-interaction-terms-and-tweedie-regression-rpes3hazq) · Towards Data Science · 0 upvotes · 0 comments --- Tags: [#python](https://daily.dev/tags/python) [View this post on daily.dev](https://daily.dev/posts/product-experimentation-with-instrumental-variables-unconfounding-llm-routing-decisions-in-python-dxangqznt) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Product Experimentation with Instrumental Variables: Unconfounding LLM Routing Decisions in Python","url":"https://daily.dev/posts/product-experimentation-with-instrumental-variables-unconfounding-llm-routing-decisions-in-python-dxangqznt","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/product-experimentation-with-instrumental-variables-unconfounding-llm-routing-decisions-in-python-dxangqznt"},"datePublished":"2026-07-29T16:20:36.803Z","dateModified":"2026-07-29T16:21:00.530Z","description":"Standard regression is fundamentally biased when measuring LLM routing quality because routing rules correlate with query complexity, a confounder that...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/d86bd18199c6ce243091dd4d83a2c4ff?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/d86bd18199c6ce243091dd4d83a2c4ff?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"freeCodeCamp","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"freeCodeCamp","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1628412854/logos/freecodecamp","url":"https://daily.dev/sources/freecodecamp"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/product-experimentation-with-instrumental-variables-unconfounding-llm-routing-decisions-in-python-dxangqznt","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":2},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"python","timeRequired":"PT21M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"freeCodeCamp","item":"https://daily.dev/sources/freecodecamp"},{"@type":"ListItem","position":3,"name":"Product Experimentation with Instrumental Variables: Unconfounding LLM Routing Decisions in Python"}]} ```