Spurious correlations are not exclusive to large, high-dimensional datasets — small studies are equally susceptible. Using the geometry of Pearson's correlation coefficient, this piece explains why sample correlations are almost always nonzero even when the true population correlation is zero. Centering data constrains vectors to an (n-1)-dimensional hyperplane, and normalization places them on a unit sphere; the correlation then equals the cosine of the angle between two points on that sphere. Under Gaussian assumptions, rotational invariance yields the exact sampling distribution of the correlation, which is U-shaped for n=3, uniform for n=4, and converges to a normal distribution as n grows. Practical implications include: with only 3 subjects, correlations near ±1 are more likely than correlations near 0; with 10 subjects, there is a 25% chance of |r|>0.4 even under independence; and the exact Gaussian null distribution is surprisingly robust for symmetric non-normal data. The key takeaway is to always examine sample size before interpreting any reported correlation.

13m read timeFrom towardsdatascience.com
Post cover image
Table of contents
The Gaussian ExperimentWhat is the sample correlation between two vectors?What about non-normal sample distributions?What does it all mean for a practitioner?The one takeawayFeedback welcome
108 Impressions