--- title: 32/60 Days System Design Questions! | daily.dev description: A scenario-based system design question about secrets management for a startup facing a SOC 2 audit. The setup reveals common anti-patterns: secrets committed... canonical: https://daily.dev/posts/32-60-days-system-design-questions--hyj2hjqy5 twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: 32/60 Days System Design Questions! | daily.dev og:description: A scenario-based system design question about secrets management for a startup facing a SOC 2 audit. The setup reveals common anti-patterns: secrets committed... og:url: https://daily.dev/posts/32-60-days-system-design-questions--hyj2hjqy5 og:image: https://api.daily.dev/og/posts/HYj2hJQy5.png og:image:alt: 32/60 Days System Design Questions! og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # 32/60 Days System Design Questions! **[Joud Awad](https://daily.dev/sources/iac4jsbu0lv8wbsc85fsh)** · [@joudawad](https://daily.dev/joudawad) · 2 min read · 46 upvotes · 13 comments ## Summary A scenario-based system design question about secrets management for a startup facing a SOC 2 audit. The setup reveals common anti-patterns: secrets committed to git, hardcoded in Lambda env vars, and stored in Slack messages with no rotation or audit trail. Four approaches are presented for fixing this without downtime: AWS Secrets Manager, HashiCorp Vault, CI/CD-injected environment variables, and KMS-encrypted secrets in a custom database. Readers are invited to pick an option and discuss trade-offs. ## Content Your startup just got its first SOC 2 audit. The auditor asks: "Where are your database passwords, API keys, and service tokens stored?" Your senior engineer goes quiet. Turns out half of them are in .env files committed to git 18 months ago. Three are hardcoded in Lambda environment variables. One is in a Slack message from 2023. You have 6 services in production, 4 environments, and zero rotation policy. Here's the setup: • NestJS API → Postgres (password in env var) • NestJS API → Stripe (API key in env var) • Background workers → SQS, S3 (AWS credentials in env var) • 3rd-party webhooks → HMAC secrets in env var • Zero rotation. Zero audit trail. Zero centralized access control. You need to fix this. And you can't take downtime. **A)** Move everything to AWS Secrets Manager — SDK calls at runtime, IAM controls access, auto-rotation built in. **B)** Use HashiCorp Vault — dynamic secrets, fine-grained policies, works across any cloud or on-prem. **C)** Use environment variables injected at deploy time via CI/CD — secrets stored in GitHub Actions / GitLab CI secrets vault, never touch disk. **D)** Encrypt secrets with KMS and store ciphertext in your own database — decrypt at runtime, full control. All four are used in production at real companies. Pick one — A, B, C, or D — and tell me why. I'll drop the full breakdown in the comments. If your team is having this argument right now, share this post. Someone needs to see it. Drop your answer 👇 _#30DaysOfSystemDesign #SystemDesign #BackendEngineering #CloudArchitecture_ ## Community discussion Top comments from developers on daily.dev. **@joudawad** · 6 upvotes > **B — HashiCorp Vault (senior engineer trap)** > > > Vault is _more powerful_ — dynamic secrets (a fresh Postgres credential per-request that expires in 1h) is one of the most elegant patterns in production infra. > > > But Vault is infrastructure you own and operate. You need a Raft cluster (3+ nodes), a storage backend, unsealing procedures, DR planning, and an ops team that knows Vault. If Vault goes down, secrets become inaccessible — services fail. > > > **When Vault wins:** Multi-cloud, on-prem requirements, dynamic secrets at scale, or you have a dedicated infra team. If you're AWS-native under 50... **@joudawad** · 4 upvotes > **A — AWS Secrets Manager (correct)** > > > You're already AWS-native (Lambda, SQS, S3). Secrets Manager gives you: > > > • Centralized encrypted storage (KMS-backed) > > • IAM-based access control per secret per service > > • Native auto-rotation for RDS + custom Lambda rotators for everything else > > • Every access logged in CloudTrail → exactly what your SOC 2 auditor wants > > > Migration is zero-downtime: swap env vars for getSecretValue() calls, deploy, rotate the old creds after confirming the new ones work. Cost: ~$0.40/secret/month. For 20 secrets that's $8/month. > > >  ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"DiscussionForumPosting","mainEntityOfPage":"https://daily.dev/posts/32-60-days-system-design-questions--hyj2hjqy5","headline":"32/60 Days System Design Questions!","text":"A scenario-based system design question about secrets management for a startup facing a SOC 2 audit. The setup reveals common anti-patterns: secrets committed to git, hardcoded in Lambda env vars, and stored in Slack messages with no rotation or audit trail. Four approaches are presented for fixing this without downtime: AWS Secrets Manager, HashiCorp Vault, CI/CD-injected environment variables, and KMS-encrypted secrets in a custom database. Readers are invited to pick an option and discuss trade-offs.","url":"https://daily.dev/posts/32-60-days-system-design-questions--hyj2hjqy5","datePublished":"2026-06-07T18:17:33.425Z","dateModified":"2026-06-07T18:20:44.719Z","author":{"@type":"Person","name":"Joud Awad","url":"https://daily.dev/joudawad","image":"https://media.daily.dev/image/upload/s--dOB9RaXY--/f_auto/v1773320801/avatars/avatar_iaC4JsBU0lV8wBsc85fSh?_a=BAMAMiiu0","description":"Principal Solution Architecture ","worksFor":{"@type":"Organization","name":"Metalab","logo":"https://www.google.com/s2/favicons?domain=metalab.com&sz=128"},"interactionStatistic":{"@type":"InteractionCounter","interactionType":{"@type":"EndorseAction"},"userInteractionCount":81090}},"image":"https://media.daily.dev/image/upload/s--1vcBntj_--/f_auto/v1780856259/posts/HYj2hJQy5?_a=BAMAMiWQ0","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":46},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":13}],"comment":[{"@type":"Comment","text":"B — HashiCorp Vault (senior engineer trap)\nVault is more powerful — dynamic secrets (a fresh Postgres credential per-request that expires in 1h) is one of the most elegant patterns in production infra.\nBut Vault is infrastructure you own and operate. You need a Raft cluster (3+ nodes), a storage backend, unsealing procedures, DR planning, and an ops team that knows Vault. If Vault goes down, secrets become inaccessible — services fail.\nWhen Vault wins: Multi-cloud, on-prem requirements, dynamic secrets at scale, or you have a dedicated infra team. If you’re AWS-native under 50 engineers — Secrets Manager first, always.","datePublished":"2026-06-07T18:18:07.792Z","url":"https://daily.dev/posts/HYj2hJQy5#c-t9edMSJBI","author":{"@type":"Person","name":"Joud Awad","url":"https://daily.dev/joudawad","image":"https://media.daily.dev/image/upload/s--dOB9RaXY--/f_auto/v1773320801/avatars/avatar_iaC4JsBU0lV8wBsc85fSh?_a=BAMAMiiu0"},"interactionStatistic":{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":6}},{"@type":"Comment","text":"A — AWS Secrets Manager (correct)\nYou’re already AWS-native (Lambda, SQS, S3). Secrets Manager gives you:\n• Centralized encrypted storage (KMS-backed)\n• IAM-based access control per secret per service\n• Native auto-rotation for RDS + custom Lambda rotators for everything else\n• Every access logged in CloudTrail → exactly what your SOC 2 auditor wants\nMigration is zero-downtime: swap env vars for getSecretValue() calls, deploy, rotate the old creds after confirming the new ones work. Cost: ~$0.40/secret/month. For 20 secrets that’s $8/month.","datePublished":"2026-06-07T18:18:04.945Z","url":"https://daily.dev/posts/HYj2hJQy5#c-9HrF2K1qm","author":{"@type":"Person","name":"Joud Awad","url":"https://daily.dev/joudawad","image":"https://media.daily.dev/image/upload/s--dOB9RaXY--/f_auto/v1773320801/avatars/avatar_iaC4JsBU0lV8wBsc85fSh?_a=BAMAMiiu0"},"interactionStatistic":{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":4}},{"@type":"Comment","text":"C — CI/CD Injected Env Vars (partially right, wrong scope)\nGitHub Actions / GitLab CI secrets are the right place for build-time and deploy-time secrets — Docker Hub creds, Terraform tokens, deploy keys.\nThey’re wrong for runtime secrets — the DB password your app needs at 3 AM, the Stripe key for a live webhook. Those get injected as env vars into ECS task definitions or Lambda config — harder to rotate atomically, no audit trail on runtime access.\nC complements A or B. It doesn’t replace them.","datePublished":"2026-06-07T18:18:12.197Z","url":"https://daily.dev/posts/HYj2hJQy5#c-oRCVGF97K","author":{"@type":"Person","name":"Joud Awad","url":"https://daily.dev/joudawad","image":"https://media.daily.dev/image/upload/s--dOB9RaXY--/f_auto/v1773320801/avatars/avatar_iaC4JsBU0lV8wBsc85fSh?_a=BAMAMiiu0"},"interactionStatistic":{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":2}},{"@type":"Comment","text":"D — DIY KMS + own database (wrong)\nThis is reinventing Secrets Manager badly. You now own: encryption key management, rotation logic, access control, audit logging, and a database that itself needs credentials to connect to.\nThe meta-problem: what secret protects the secret store? You end up with a bootstrapping problem that managed secret stores are specifically designed to solve. This is what teams built in 2012 before managed options existed.","datePublished":"2026-06-07T18:18:16.588Z","url":"https://daily.dev/posts/HYj2hJQy5#c-W3i4YhL5I","author":{"@type":"Person","name":"Joud Awad","url":"https://daily.dev/joudawad","image":"https://media.daily.dev/image/upload/s--dOB9RaXY--/f_auto/v1773320801/avatars/avatar_iaC4JsBU0lV8wBsc85fSh?_a=BAMAMiiu0"},"interactionStatistic":{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":2}},{"@type":"Comment","text":"Hi, I like that you try to get people to think about architecture. A personal note here, all of the options are missing essential architecture details. How are your Nodejs app and database hosted? On EC2 on ECS? Are we using plain RDS? Are we using an rds instance or cluster? Does your company have Hashicorp Cloud or do you have to host and setup your own Vault instance? The latter leads to a lot of follow ups.\nRegarding rds, do not use secrets for connecting from e.g. EC2 or ECS with RDS. Use IAM auth instead and create a custom db user to avoid admin user usage. This is sec default practice.\nDo you have multiple envs and want to follow 12 factor apps? Build once and deploy many. Meaning depending on your software stack you use, add the runtime config at deploy time, not build time. So a combo of different solutions might be better than a plain single option.","datePublished":"2026-06-09T04:32:37.949Z","url":"https://daily.dev/posts/HYj2hJQy5#c-CNakdbVB4","author":{"@type":"Person","name":"Lars Kinder","url":"https://daily.dev/larskinder","image":"https://lh3.googleusercontent.com/a/ACg8ocK8gMCtCwZpyw_ZANKOoco5XrVRJRQtCKpqZcouwaWL1zEpzQ=s96-c"},"interactionStatistic":{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":2}}],"isPartOf":{"@type":"WebPage","url":"https://daily.dev/sources/iac4jsbu0lv8wbsc85fsh","name":"Joud Awad"}} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Joud Awad","item":"https://daily.dev/sources/iac4jsbu0lv8wbsc85fsh"},{"@type":"ListItem","position":3,"name":"32/60 Days System Design Questions!"}]} ```