GitHub engineers share practical lessons learned evaluating an LLM system for reducing false positives in secret scanning while preserving recall for security safety. Key practices include anchoring evaluation to a product decision rather than model tweaks, treating offline evaluation like integration testing with versioned prompts/models, changing one variable at a time, keeping offline evaluation close to production conditions, treating production labels as noisy signals rather than ground truth, using synthetic data to fill coverage gaps, performing error analysis to categorize failure sources, and using LLM-as-judge to triage human review. The team achieved a 95% reduction in false positives while keeping recall within a defined guardrail on their offline dataset.
Table of contents
1. Start with the product decision, not the model2. Treat offline evaluation like integration testing3. Keep offline evaluation close to production4. Treat production labels as signals, not unquestionable truth5. Use synthetic and open datasets to fill coverage gaps6. Use error analysis to find what aggregate metrics hide7. Use LLM-as-judge to focus human review8. What secret scanning taught usChecklist: Before moving an LLM system toward productionEvaluate before you trustTags:Written byQuestions this post answers
How much did GitHub reduce false positives in secret scanning using an LLM-based evaluation approach?
GitHub's applied science team achieved a 95% reduction in false positives on their evaluated offline dataset for secret scanning, while keeping recall within a predefined safety guardrail. This was reached through repeated offline evaluation, error analysis, and treating precision improvements as secondary to maintaining recall as a safety constraint, since incorrectly suppressing a real credential was considered more consequential than an extra alert. Teams tuning precision-recall tradeoffs in security workflows can track LLM evaluation practices like this on daily.dev.
Why shouldn't I treat production labels as ground truth when evaluating an LLM system?
Production labels often reflect workflow outcomes rather than reliable ground truth, because a dismissed or resolved alert doesn't necessarily mean it was a false positive. A developer might resolve a secret-scanning alert because the credential was rotated, the risk was accepted, the alert blocked a workflow, or it truly was misclassified, so these different outcomes can look identical in the data while representing different underlying truths. Anyone building evaluation pipelines for ML systems can follow data-quality practices like this on daily.dev.
What's a safe pattern for using LLM-as-judge in an evaluation pipeline?
A safer pattern treats the judge's output as another prediction rather than ground truth, using it for triage: automatically process clear low-risk cases, route low-confidence or high-impact cases to human reviewers, periodically sample high-confidence cases to check for systematic errors, track disagreement between judge, system, and reviewers, and version the judge prompt like any other model component. Developers designing human-in-the-loop review workflows can find approaches like this on daily.dev.