--- title: "Issue #139 - Bayesian Models: How Sure Are You?" url: https://daily.dev/posts/issue-139---bayesian-models-how-sure-are-you--9bdxglcjy source_url: https://mlpills.substack.com/p/issue-139-bayesian-models-how-sure type: article source: "Machine Learning Pills" published: 2026-08-12T15:09:40.490Z updated: 2026-08-12T15:10:12.644Z tags: ["python", "data-science", "ab-testing"] reading_time: 10 upvotes: 0 comments: 0 language: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Issue #139 - Bayesian Models: How Sure Are You? **[Machine Learning Pills](https://daily.dev/sources/mlpills)** · 10 min read · 0 upvotes · 0 comments ## Summary An explanation of Bayesian modeling as an alternative to point-estimate machine learning, showing how priors, likelihoods, and posteriors give you a full distribution of plausible values rather than a single number. Uses conversion-rate examples with scipy's beta distribution to compute credible intervals and expected loss for A/B test decisions, contrasts credible intervals with frequentist confidence intervals, and introduces hierarchical models for 'borrowing strength' across small sample groups using PyMC. Covers conjugate priors (Beta, Gamma, Normal, Dirichlet), practical prior-setting rules, and when to move from closed-form solutions to samplers like PyMC, NumPyro, or Stan. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How do I compute a Bayesian credible interval for a conversion rate using scipy? Use a Beta posterior with scipy.stats: posterior = stats.beta(1 + conversions, 1 + visitors - conversions), then posterior.mean() for the best guess and posterior.ppf([0.025, 0.975]) for the 95% credible interval. This closed-form conjugate update works because Beta is the conjugate prior for binomial success/trial data, requiring no sampling or fitting loop. _daily.dev surfaces practical Bayesian statistics techniques for developers building experimentation systems._ ### Why might an A/B test show no statistical significance (p < 0.05) but still be worth shipping? Because expected loss can favor shipping even without significance: in one paywall test, variant B (7.30% conversion) never reached p < 0.05 against variant A (5.92%), yet the expected loss from wrongly shipping B was 0.0004 conversions per impression versus 0.0143 for wrongly keeping A, a 34x difference, making the Bayesian decision clearly favor B. _Teams weighing statistical significance against business risk can track these decision frameworks on daily.dev._ ### What is shrinkage in a hierarchical Bayesian model and why does it matter for small sample groups? Shrinkage is the pull of a small group's estimate toward the overall group average, applied automatically based on how much data that group has and how different groups genuinely are. In one example, a paywall variant with 95 impressions and a raw 14.74% conversion rate was adjusted down to 10.86% after shrinkage, revealing that its apparent lead was mostly noise from a small sample. _daily.dev helps practitioners comparing hierarchical modeling approaches for noisy, low-sample experiment arms._ ## Similar posts on daily.dev - [Bayesian statistics for confused data scientists](https://daily.dev/posts/bayesian-statistics-for-confused-data-scientists-vkyv2eh5g) · Hacker News · 1 upvotes · 0 comments - [Bayesian Guardrails for AI Decisions: Measuring Uncertainty Before Automating Decisions](https://daily.dev/posts/bayesian-guardrails-for-ai-decisions-measuring-uncertainty-before-automating-decisions-bpwxolimh) · Towards Data Science · 0 upvotes · 0 comments - [Bayesian Thinking for People Who Hated Statistics](https://daily.dev/posts/bayesian-thinking-for-people-who-hated-statistics-oh83v3bw6) · Towards Data Science · 0 upvotes · 0 comments --- Tags: [#python](https://daily.dev/tags/python), [#data-science](https://daily.dev/tags/data-science), [#ab-testing](https://daily.dev/tags/ab-testing) [View this post on daily.dev](https://daily.dev/posts/issue-139---bayesian-models-how-sure-are-you--9bdxglcjy)