--- title: 🚨How one tiny SQL query nearly killed our production... description: A production outage caused by a seemingly innocent `SELECT *` query demonstrates how database queries that work fine in development can fail catastrophically... canonical: https://daily.dev/posts/how-one-tiny-sql-query-nearly-killed-our-production-system--jxaxzeqqi twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: 🚨How one tiny SQL query nearly killed our production system 🚨 | daily.dev og:description: A production outage caused by a seemingly innocent `SELECT *` query demonstrates how database queries that work fine in development can fail catastrophically... og:url: https://daily.dev/posts/how-one-tiny-sql-query-nearly-killed-our-production-system--jxaxzeqqi og:image: https://api.daily.dev/og/posts/JxaxZeQQi.png og:image:alt: 🚨How one tiny SQL query nearly killed our production system 🚨 og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # 🚨How one tiny SQL query nearly killed our production system 🚨 **[Jobs](https://daily.dev/sources/jobs)** · [@ebuz](https://daily.dev/ebuz) · 3 min read · 319 upvotes · 33 comments ## Summary A production outage caused by a seemingly innocent `SELECT *` query demonstrates how database queries that work fine in development can fail catastrophically at scale. The query pulled 720MB of data from 60,000+ records, causing memory pressure, network bottlenecks, and connection timeouts. The post provides practical solutions including selecting specific columns, implementing pagination, caching hot data, using EXPLAIN plans, monitoring slow queries, and testing with production-scale data. ## Content It was launch day. Traffic was high. Everything looked good. Then… within 10 minutes, the site went down. The root cause? A single SQL query: ```sql SELECT * FROM orders WHERE status = 'pending'; ``` In development, this query felt harmless. I only had \~500 test records. Response times were instant. But in production, things looked very different: * **60,000+ orders** in the table * Each row = \~12KB (id, customer info, shipping, items, metadata, logs) * Total payload = \~720MB pulled into memory That one query triggered a cascade of problems: 1. **Memory pressure** → Database engine tried to hold 700MB+ in RAM. 2. **Network bottleneck** → Transferring hundreds of MB across the wire = 30–40s delays. 3. **Connection lock** → Each slow query kept connections busy for too long. 4. **Concurrency disaster** → 100 users × 700MB queries = DB server saturated, new requests timed out. All because of a single `*`. --- ### Why `SELECT *` is dangerous in production * **Column bloat**: You pull back data you’ll never use (images, descriptions, logs). * **Unstable schemas**: If new columns are added later, they get fetched automatically → bloated queries without you noticing. * **Index inefficiency**: Query planner may not optimize properly when fetching wide rows. * **Bandwidth + memory overhead**: Every extra KB wastes CPU cycles, RAM, and network. * **Concurrency collapse**: Under load, these inefficiencies multiply exponentially. --- ### The better approach 1. **Select only what you need** ```sql SELECT id, customer_id, total_price FROM orders WHERE status = 'pending' LIMIT 50 OFFSET 0; ``` 2. **Paginate aggressively** Never load 10k+ rows in one go. Use `LIMIT` + `OFFSET` or keyset pagination. 3. **Cache hot data** Frequent queries (e.g., active products, pending orders) → push to Redis/Memcached. 4. **Use EXPLAIN** Check how your queries behave on large datasets. Don’t assume dev ≈ prod. 5. **Monitor slow queries** Enable MySQL/Postgres slow query logs. Anything >200ms deserves investigation. 6. **Test with production-like data** Your dev DB with 500 records hides problems. Mirror production scale locally or in staging. 7. **Set query timeouts** Never let a single bad query hog resources forever. --- ✅ **Before pushing a query to production — check these 5 things:** 1️⃣ **Columns** → Are you selecting only what you actually need? 2️⃣ **Pagination** → Are you limiting rows (LIMIT/OFFSET or keyset pagination)? 3️⃣ **Indexes** → Does your WHERE/JOIN condition hit an index? 4️⃣ **Scale test** → Have you tested with production-size data, not just dev seed data? 5️⃣ **Monitoring** → Is slow query logging + timeout enabled? --- 🔒 Bonus: Cache hot queries (Redis/Memcached) → cut DB load by 80%+ --- 🔑 **Takeaway:** In production, every byte counts. A single `SELECT *` might seem innocent in dev… …but at scale, it can cripple your database and take your system offline. Always code with production scale in mind. 🚀 ## Community discussion Top comments from developers on daily.dev. **@skeptiq** · 24 upvotes > Never `SELECT *` - It's elementary, my dear Watson. **@dark\_seid** · 6 upvotes > SELECT * feels harmless… until it meets production. Noted🫡 **@sqlmac** · 3 upvotes > SELECT * never hits an index, it's an automatic Table SCAN every time. **@mdshiponahammed** · 3 upvotes > Insightfull **@dawasherpa** · 2 upvotes > May be you belong in the vibe ## Similar posts on daily.dev - [Don’t just attend KubeCon \+ CloudNativeCon, Merge Forward your experience\!](https://daily.dev/posts/don-t-just-attend-kubecon-cloudnativecon-merge-forward-your-experience--l0rpp73x8) · CNCF · 1 upvotes · 0 comments - [Announcing H2 2026 KCDs](https://daily.dev/posts/announcing-h2-2026-kcds-m96goajm1) · CNCF · 1 upvotes · 0 comments - [Two months of Open Community Groups](https://daily.dev/posts/two-months-of-open-community-groups-asf52zhbs) · CNCF · 0 upvotes · 0 comments - [CNCF Unveils Schedule for KubeCon \+ CloudNativeCon Europe 2026](https://daily.dev/posts/cncf-unveils-schedule-for-kubecon-cloudnativecon-europe-2026-ikhcoa5cb) · CNCF · 2 upvotes · 0 comments - [CNCF Debuts KubeCon \+ CloudNativeCon Japan 2026 Schedule](https://daily.dev/posts/cncf-debuts-kubecon-cloudnativecon-japan-2026-schedule-xp5pyudub) · CNCF · 1 upvotes · 0 comments --- Tags: [#performance](https://daily.dev/tags/performance), [#database](https://daily.dev/tags/database), [#sql](https://daily.dev/tags/sql) [View this post on daily.dev](https://daily.dev/posts/how-one-tiny-sql-query-nearly-killed-our-production-system--jxaxzeqqi) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"DiscussionForumPosting","mainEntityOfPage":"https://daily.dev/posts/how-one-tiny-sql-query-nearly-killed-our-production-system--jxaxzeqqi","headline":"🚨How one tiny SQL query nearly killed our production system 🚨","text":"A production outage caused by a seemingly innocent `SELECT *` query demonstrates how database queries that work fine in development can fail catastrophically at scale. The query pulled 720MB of data from 60,000+ records, causing memory pressure, network bottlenecks, and connection timeouts. The post provides practical solutions including selecting specific columns, implementing pagination, caching hot data, using EXPLAIN plans, monitoring slow queries, and testing with production-scale data.","url":"https://daily.dev/posts/how-one-tiny-sql-query-nearly-killed-our-production-system--jxaxzeqqi","datePublished":"2025-08-29T02:53:11.612Z","dateModified":"2025-09-10T14:56:54.181Z","author":{"@type":"Person","name":"Taki Elias","url":"https://daily.dev/ebuz","image":"https://media.daily.dev/image/upload/v1667280113/avatars/avatar_WPc7MkTKlK9IHPGFzkjfv.jpg","description":"Self-Taught Polyglot Programmer, Learner","interactionStatistic":{"@type":"InteractionCounter","interactionType":{"@type":"EndorseAction"},"userInteractionCount":5720}},"image":"https://media.daily.dev/image/upload/s--TOE_ODim--/f_auto/v1756435993/posts/JxaxZeQQi?_a=BAMClqZW0","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":319},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":33}],"comment":[{"@type":"Comment","text":"Never SELECT * - It’s elementary, my dear Watson.","datePublished":"2025-09-02T13:37:32.377Z","url":"https://daily.dev/posts/JxaxZeQQi#c-apkksw1cU","author":{"@type":"Person","name":"The Skeptiq","url":"https://daily.dev/skeptiq","image":"https://media.daily.dev/image/upload/s--OTJ1UWI6--/f_auto/v1738074563/avatars/avatar_ic83Smu6jxYTzCXCWtIZf"},"interactionStatistic":{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":24}},{"@type":"Comment","text":"SELECT * feels harmless… until it meets production. Noted🫡","datePublished":"2025-09-28T04:12:03.998Z","url":"https://daily.dev/posts/JxaxZeQQi#c-E2Vef0kWt","author":{"@type":"Person","name":"Andile Mazibuko","url":"https://daily.dev/dark_seid","image":"https://media.daily.dev/image/upload/s--vvFG4d3---/f_auto/v1759033126/avatars/avatar_hSvzCDegNd3ifBCDgUFFr?_a=BAMAK+ZW0"},"interactionStatistic":{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":6}},{"@type":"Comment","text":"SELECT * never hits an index, it’s an automatic Table SCAN every time.","datePublished":"2025-10-06T14:18:27.617Z","url":"https://daily.dev/posts/JxaxZeQQi#c-gW6Htqh7t","author":{"@type":"Person","name":"SQLMac","url":"https://daily.dev/sqlmac","image":"https://avatars.githubusercontent.com/u/1228957?v=4"},"interactionStatistic":{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":3}},{"@type":"Comment","text":"Insightfull","datePublished":"2025-09-04T10:40:07.562Z","url":"https://daily.dev/posts/JxaxZeQQi#c-wEAZbbIkF","author":{"@type":"Person","name":"md shipon Ahammed","url":"https://daily.dev/mdshiponahammed","image":"https://lh3.googleusercontent.com/a/ACg8ocLahFh6aHSU0dOUGwrnnoBBtF8i_4eW8qbTdVh_kzlDk-sv0ng=s96-c"},"interactionStatistic":{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":3}},{"@type":"Comment","text":"May be you belong in the vibe","datePublished":"2025-10-14T13:21:46.769Z","url":"https://daily.dev/posts/JxaxZeQQi#c-4CMyO2ZNn","author":{"@type":"Person","name":"Dawa Sherpa","url":"https://daily.dev/dawasherpa","image":"https://media.daily.dev/image/upload/s--o_DgY2EO--/f_auto/v1727105332/avatars/avatar_jHemDuXHdEVDeyOzTMtPl"},"interactionStatistic":{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":2}}],"isPartOf":{"@type":"WebPage","url":"https://daily.dev/squads/jobs","name":"Jobs"}} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Jobs","item":"https://daily.dev/squads/jobs"},{"@type":"ListItem","position":3,"name":"🚨How one tiny SQL query nearly killed our production system 🚨"}]} ```