An explanation of Bayesian modeling as an alternative to point-estimate machine learning, showing how priors, likelihoods, and posteriors give you a full distribution of plausible values rather than a single number. Uses conversion-rate examples with scipy's beta distribution to compute credible intervals and expected loss for A/B test decisions, contrasts credible intervals with frequentist confidence intervals, and introduces hierarchical models for 'borrowing strength' across small sample groups using PyMC. Covers conjugate priors (Beta, Gamma, Normal, Dirichlet), practical prior-setting rules, and when to move from closed-form solutions to samplers like PyMC, NumPyro, or Stan.
Table of contents
Same number, very different confidenceWhy this range is the one people think they are gettingTurning the curve into a decisionPriors, and why they are less scary than they soundThe technique that earns its keep: borrowing strengthWhen you need a real samplerQuestions this post answers
How do I compute a Bayesian credible interval for a conversion rate using scipy?
Use a Beta posterior with scipy.stats: posterior = stats.beta(1 + conversions, 1 + visitors - conversions), then posterior.mean() for the best guess and posterior.ppf([0.025, 0.975]) for the 95% credible interval. This closed-form conjugate update works because Beta is the conjugate prior for binomial success/trial data, requiring no sampling or fitting loop. daily.dev surfaces practical Bayesian statistics techniques for developers building experimentation systems.
Why might an A/B test show no statistical significance (p < 0.05) but still be worth shipping?
Because expected loss can favor shipping even without significance: in one paywall test, variant B (7.30% conversion) never reached p < 0.05 against variant A (5.92%), yet the expected loss from wrongly shipping B was 0.0004 conversions per impression versus 0.0143 for wrongly keeping A, a 34x difference, making the Bayesian decision clearly favor B. Teams weighing statistical significance against business risk can track these decision frameworks on daily.dev.
What is shrinkage in a hierarchical Bayesian model and why does it matter for small sample groups?
Shrinkage is the pull of a small group's estimate toward the overall group average, applied automatically based on how much data that group has and how different groups genuinely are. In one example, a paywall variant with 95 impressions and a raw 14.74% conversion rate was adjusted down to 10.86% after shrinkage, revealing that its apparent lead was mostly noise from a small sample. daily.dev helps practitioners comparing hierarchical modeling approaches for noisy, low-sample experiment arms.