--- title: "7/30 Days System Design Questions!" url: https://daily.dev/posts/7-30-days-system-design-questions--kff1zomy4 source_url: https://daily.dev/posts/7-30-days-system-design-questions--kff1zomy4 type: freeform source: "Joud Awad" author: "Joud Awad" published: 2026-05-12T16:02:46.399Z updated: 2026-05-12T16:04:43.539Z tags: ["aws", "career", "architecture", "distributed-systems"] reading_time: 2 upvotes: 84 comments: 7 language: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # 7/30 Days System Design Questions! **[Joud Awad](https://daily.dev/sources/iac4jsbu0lv8wbsc85fsh)** · [@joudawad](https://daily.dev/joudawad) · 2 min read · 84 upvotes · 7 comments ## Summary A production incident scenario where 47 orders got stuck in a 'cancelled' state with no 'created' record, caused by out-of-order message processing in a standard SQS queue with 5 parallel consumers. Three events (created, paid, cancelled) fired within 400ms were picked up by different workers simultaneously, causing the downstream state machine to process 'cancelled' before 'created'. Four solution patterns are presented: SQS FIFO with MessageGroupId, a reorder buffer in the consumer, a Saga pattern, or making the state machine idempotent and order-agnostic. Readers are asked to pick the best solution and explain their reasoning. ## Content Your order service publishes 3 events per order: created, paid, cancelled. Standard SQS queue, 5 consumers, ~2K orders/minute at peak. Last night, prod alerted: 47 orders stuck in "cancelled" state with no "created" record. Customers got refunded for orders that were never marked as placed. Finance is not happy. You dig in. The order was placed, paid, and cancelled within 400ms. Three messages hit SQS in order. Three consumers picked them up in parallel. "cancelled" got processed first. State machine rejected "created" as invalid. Data is now wrong. Here's the setup: OrderService → SQS (standard) → 5 workers → Postgres Events: order.created → order.paid → order.cancelled Problem: workers process in parallel, SQS standard doesn't guarantee order, downstream state machine breaks when events arrive out of sequence. You have until Monday. What do you do? A) Switch to SQS FIFO + MessageGroupId per order_id — let AWS handle ordering per order. B) Keep standard SQS, add a sequence number + reorder buffer in the consumer — hold out-of-order events until the earlier ones arrive. C) Replace the event stream with a Saga — each step waits for the previous step's completion signal before firing the next. D) Add an event version/timestamp and make the state machine idempotent + order-agnostic — reject stale transitions, accept any arrival order. All four are patterns I've seen in real production systems. Only one actually fits this problem. One of them is the trap that gets senior engineers on the whiteboard. Pick one — A, B, C, or D — and tell me why. Full breakdown in the comments. If your team has argued about ordering guarantees at 2am, forward this to them. That debate is where the real learning happens. Drop your answer 👇 #30DaysOfSystemDesign #Day7 #SystemDesign #DistributedSystems ## Community discussion Top comments from developers on daily.dev. **@joudawad** · 5 upvotes > **✅ Answer: A — SQS FIFO + MessageGroupId per order_id** > > > **Why A wins:** > > You don't need global ordering — you need _per-order_ ordering. Set MessageGroupId = order_id. FIFO guarantees messages with the same group ID are delivered in exact send order, with only one consumer processing that group at a time. Your 2K orders/min still fan out across 5 workers — throughput is unchanged. Order #12345's created → paid → cancelled always lands in sequence. Cost: a few ms latency. Ship it Monday. > > ![ChatGPT Image May 12, 2026, 07_01_58 PM... **@joudawad** · 2 upvotes > **Why B is the trap (reorder buffer):** > > Fools senior engineers on the whiteboard. You'd end up building: in-memory buffer per order_id + TTL logic + dead-letter path for gaps + crash recovery + stuck buffer monitoring. You just rebuilt FIFO in app code. With more bugs. And you own every edge case. B is valid only when you _can't_ use FIFO (e.g. Kafka with a taken partition key). That's not this situation. **@joudawad** · 2 upvotes > **Why C is wrong (Saga):** > > Saga is for long-running distributed transactions with compensating actions (book flight → charge card → reserve hotel). Your problem is a _state stream for one entity_ — not a multi-service transaction. Forcing it into a Saga turns async events into sync RPC, increases coupling, and still doesn't fix ordering. **@joudawad** · 2 upvotes > **Why D is wrong (event versioning):** > > Partially works — but you'd _drop_ legitimate events. "cancelled" arrives first, moves state forward, then "created" gets rejected. DB says order is cancelled with no creation record. To make D work correctly you need full event sourcing (6-month rewrite, not a Monday fix). **@joudawad** · 1 upvotes > Also, it would mean a lot to me if you could support my content and stay in touch 🙏 > > - YouTube: [https://www.youtube.com/@system-design-lab](https://www.youtube.com/@system-design-lab) > - LinkedIn: [https://www.linkedin.com/in/joud-awad/](https://www.linkedin.com/in/joud-awad/) > - Medium Blog: [https://joudwawad.medium.com/](https://joudwawad.medium.com/) > - Substack: [https://joudawad.substack.com/](https://joudawad.substack.com/) --- Tags: [#aws](https://daily.dev/tags/aws), [#career](https://daily.dev/tags/career), [#architecture](https://daily.dev/tags/architecture), [#distributed-systems](https://daily.dev/tags/distributed-systems) [View this post on daily.dev](https://daily.dev/posts/7-30-days-system-design-questions--kff1zomy4)