--- title: The hardest test suite I ever built - a pytest case... description: [EuroPython 2026 - S2 on 2026-07-15] 🎤 *The hardest test suite I ever built - a pytest case study by Grzegorz Kocjan* 🔗... canonical: https://daily.dev/posts/the-hardest-test-suite-i-ever-built---a-pytest-case-study---grzegorz-kocjan-bgxohoqbv twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: The hardest test suite I ever built - a pytest case study - Grzegorz Kocjan | daily.dev og:description: [EuroPython 2026 - S2 on 2026-07-15] 🎤 *The hardest test suite I ever built - a pytest case study by Grzegorz Kocjan* 🔗... og:url: https://daily.dev/posts/the-hardest-test-suite-i-ever-built---a-pytest-case-study---grzegorz-kocjan-bgxohoqbv og:image: https://api.daily.dev/og/posts/bgxOHoQbv.png og:image:alt: The hardest test suite I ever built - a pytest case study - Grzegorz Kocjan og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # The hardest test suite I ever built - a pytest case study - Grzegorz Kocjan **[EuroPython Conference](https://daily.dev/sources/europython)** · 30 min read · 0 upvotes · 0 comments ## Summary [EuroPython 2026 - S2 on 2026-07-15] 🎤 *The hardest test suite I ever built - a pytest case study by Grzegorz Kocjan* 🔗 https://ep2026.europython.eu/session/the-hardest-test-suite-i-ever-built-a-pytest-case-study 📝 Abstract: For years, this real-time video system had no tests. Every change produced unpredictable side effects. Accuracy drifted. Production incidents and escalations followed. The only “verification” was manual inspection and hope. When I joined the project, this was the reality - and building a proper integration test suite became my first priority. In this talk, I’ll share how I designed and evolved the hardest integration test suite of my career using pytest - and kept it readable. The system processed live streams in production. It was non-deterministic. Individual detections were only 80–90% accurate. For testing, we replayed recorded scenarios to make system behavior observable and comparable across runs. But binary assertions were not enough. A single failed event did not mean the whole system was broken - but we needed a way to measure when it actually was. Instead of writing one massive test, I built a layered architecture: - dual parametrization - recording scope and event scope - orchestration in fixtures - assertions in tiny, single-purpose tests - statistics collection during execution - end-of-run aggregation that summarizes system accuracy The result was a suite that could detect regressions in model changes and produce reproducible evidence - HTML reports, structured dumps, and a summary statistics file. This talk explores how far pytest can be stretched beyond unit tests - into a framework for architecting complex integration systems. --- This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License: https://creativecommons.org/licenses/by-nc-sa/4.0/ ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: --- Tags: [#python](https://daily.dev/tags/python), [#conference](https://daily.dev/tags/conference) [View this post on daily.dev](https://daily.dev/posts/the-hardest-test-suite-i-ever-built---a-pytest-case-study---grzegorz-kocjan-bgxohoqbv) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"The hardest test suite I ever built - a pytest case study - Grzegorz Kocjan","url":"https://daily.dev/posts/the-hardest-test-suite-i-ever-built---a-pytest-case-study---grzegorz-kocjan-bgxohoqbv","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/the-hardest-test-suite-i-ever-built---a-pytest-case-study---grzegorz-kocjan-bgxohoqbv"},"datePublished":"2026-08-19T10:14:23.884Z","dateModified":"2026-08-19T12:02:08.755Z","description":"[EuroPython 2026 - S2 on 2026-07-15]\n🎤 *The hardest test suite I ever built - a pytest case study by Grzegorz Kocjan* \n🔗...","image":"https://i.ytimg.com/vi/uH9hsF-n8ko/sddefault.jpg","thumbnailUrl":"https://i.ytimg.com/vi/uH9hsF-n8ko/sddefault.jpg","isAccessibleForFree":true,"articleSection":"EuroPython Conference","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"EuroPython Conference","logo":"https://media.daily.dev/image/upload/s--qu8RO-lH--/f_auto,q_auto/v1777800764/logos/europython?_a=BAMAMiWQ0","url":"https://daily.dev/sources/europython"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/the-hardest-test-suite-i-ever-built---a-pytest-case-study---grzegorz-kocjan-bgxohoqbv","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"python,conference","timeRequired":"PT30M","video":{"@type":"VideoObject","name":"The hardest test suite I ever built - a pytest case study - Grzegorz Kocjan","description":"[EuroPython 2026 - S2 on 2026-07-15]\n🎤 *The hardest test suite I ever built - a pytest case study by Grzegorz Kocjan* \n🔗...","thumbnailUrl":"https://i.ytimg.com/vi/uH9hsF-n8ko/sddefault.jpg","uploadDate":"2026-08-19T10:14:23.884Z","duration":"PT30M","url":"https://api.daily.dev/r/bgxOHoQbv","embedUrl":"https://www.youtube.com/embed/uH9hsF-n8ko"}} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"EuroPython Conference","item":"https://daily.dev/sources/europython"},{"@type":"ListItem","position":3,"name":"The hardest test suite I ever built - a pytest case study - Grzegorz Kocjan"}]} ```