--- title: Have you ever underestimated Big O in background processing? description: A real-world lesson about Big O complexity in background processing: a nightly cleanup job that scanned all user files started fast but grew unbounded with... canonical: https://daily.dev/posts/have-you-ever-underestimated-big-o-in-background-processing--a08dbvk1v twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Have you ever underestimated Big O in background processing? | daily.dev og:description: A real-world lesson about Big O complexity in background processing: a nightly cleanup job that scanned all user files started fast but grew unbounded with... og:url: https://daily.dev/posts/have-you-ever-underestimated-big-o-in-background-processing--a08dbvk1v og:image: https://api.daily.dev/og/posts/A08DBvK1v.png og:image:alt: Have you ever underestimated Big O in background processing? og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Have you ever underestimated Big O in background processing? **[Rebeloper](https://daily.dev/sources/rebeloper)** · 1 min read · 0 upvotes · 0 comments ## Summary A real-world lesson about Big O complexity in background processing: a nightly cleanup job that scanned all user files started fast but grew unbounded with storage, eventually overlapping with user sessions and causing slowdowns. The fix was switching from full scans to incremental dirty-file tracking, eliminating the O(n) growth problem. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: --- [View this post on daily.dev](https://daily.dev/posts/have-you-ever-underestimated-big-o-in-background-processing--a08dbvk1v) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Have you ever underestimated Big O in background processing?","url":"https://daily.dev/posts/have-you-ever-underestimated-big-o-in-background-processing--a08dbvk1v","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/have-you-ever-underestimated-big-o-in-background-processing--a08dbvk1v"},"datePublished":"2026-03-02T14:29:20.016Z","dateModified":"2026-03-02T14:29:38.032Z","description":"A real-world lesson about Big O complexity in background processing: a nightly cleanup job that scanned all user files started fast but grew unbounded with...","image":"https://i.ytimg.com/vi/rUOUEhTzCVE/sddefault.jpg","thumbnailUrl":"https://i.ytimg.com/vi/rUOUEhTzCVE/sddefault.jpg","isAccessibleForFree":true,"articleSection":"Rebeloper","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Rebeloper","logo":"https://media.daily.dev/image/upload/s--W0sZL8K4--/f_auto/v1726990075/logos/rebeloper","url":"https://daily.dev/sources/rebeloper"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/have-you-ever-underestimated-big-o-in-background-processing--a08dbvk1v","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"","timeRequired":"PT1M","video":{"@type":"VideoObject","name":"Have you ever underestimated Big O in background processing?","description":"A real-world lesson about Big O complexity in background processing: a nightly cleanup job that scanned all user files started fast but grew unbounded with...","thumbnailUrl":"https://i.ytimg.com/vi/rUOUEhTzCVE/sddefault.jpg","uploadDate":"2026-03-02T14:29:20.016Z","duration":"PT1M","url":"https://api.daily.dev/r/A08DBvK1v","embedUrl":"https://www.youtube.com/embed/rUOUEhTzCVE"}} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Rebeloper","item":"https://daily.dev/sources/rebeloper"},{"@type":"ListItem","position":3,"name":"Have you ever underestimated Big O in background processing?"}]} ```