A podcast episode exploring how production logs can be repurposed as testing assets using AI. Guest Tanvi discusses Log Miner, an open-source Python tool that ingests logs from sources like DataDog, Elastic, or CSV files, clusters similar event sequences, and generates Gherkin-format regression test cases. The tool uses Hugging Face APIs and transformer models to map log events to meaningful test scenarios while hashing PII data for security. A real banking example illustrates how a transaction sequence bug — undetectable in lower environments — was discovered via production logs. The conversation also covers testing non-deterministic AI systems, the importance of behavioral testing over input/output validation, and why human judgment remains essential in high-stakes domains like finance and healthcare.

30m watch time
3 Impressions