A deep dive into the mathematics and tradeoffs of binary-outcome tests (pass/fail, yes/no). Using a fictional shibboleth-detection drone system and a medical diagnosis scenario, the piece walks through false positives, false negatives, base rate effects (the false positive paradox), and how to assign monetary values to outcomes to optimize a classification threshold analytically. It also covers the 'idiot light' problem — how reducing continuous data to a binary signal loses useful information — and introduces triage-style three-outcome classification as a practical middle ground. Python/scipy code is provided throughout to compute Gaussian distributions, confusion matrices, and optimal thresholds.
