--- title: Best AWS S3 posts of 2024 | daily.dev description: The most upvoted AWS S3 posts of 2024, curated by the daily.dev community. canonical: https://daily.dev/tags/aws-s3/best-of/2024 twitter:card: summary_large_image twitter:site: @dailydotdev og:url: https://daily.dev/tags/aws-s3/best-of/2024 og:type: website og:site_name: daily.dev og:title: Best AWS S3 posts of 2024 | daily.dev og:description: The most upvoted AWS S3 posts of 2024, curated by the daily.dev community. og:image: https://media.daily.dev/image/upload/s--VAY5ToZt--/f_auto/v1724209435/public/daily.dev%20-%20open%20graph --- # Best of AWS S3 — 2024 1. 1 [](/posts/how-i-created-an-aws-system-to-crush-the-architect-exam-uqvrx0aio "How I created an AWS System to crush the Architect exam") Article Code Like A Girl · 2y How I created an AWS System to crush the Architect exam The post shares insights on building a notification system using AWS services to help revise for the AWS Solutions Architect Certification. The system sends PowerPoint slides as WhatsApp messages using Lambda, S3, EventBridge, and Twilio, assisting in timely and structured revision. The architecture is scalable and can be used for any revision needs. 160 2 2. 2 [](/posts/cloudlena-s3manager-a-web-gui-for-your-s3-buckets-kb727nram "cloudlena/s3manager: A Web GUI for your S3 buckets") Article Community Picks · 2y cloudlena/s3manager: A Web GUI for your S3 buckets s3manager is a Web GUI written in Go for managing S3 buckets from any provider. It allows users to list buckets, create new ones, upload and download objects, and delete objects. The application is configurable via environment variables and can be run locally, with Docker, or deployed to a Kubernetes cluster. It supports SSL, various signature types, IAM roles, and server-side encryption. 65 2 3. 3 [](/posts/s3-is-showing-its-age-cnip6rya8 "S3 Is Showing Its Age") Article Substack · 2y S3 Is Showing Its Age S3 is falling behind its competitors in terms of features such as compare-and-swap operation, multi-region buckets, and object appends. These missing features are becoming a bigger problem as new infrastructure relies on object storage as the primary persistence layer. The decision to abandon or embrace S3 depends on the system's use cases and design goals. Preconditions on S3 and a unified S3Express One Zone API would make this decision easier. 54 1 4. 4 [](/posts/i-wish-i-knew-this-one-thing-before-learning-aws-jqjhmvy6n "I Wish I Knew This One Thing Before Learning AWS") Video Be A Better Dev · 2y I Wish I Knew This One Thing Before Learning AWS Understanding the core services in AWS, such as EC2, S3, SQS, SNS, and DynamoDB, is essential before diving into other services. These foundational services form the basis upon which other, more advanced services are built. Having a strong understanding of these basics will facilitate learning and using more complex AWS offerings. 53 5. 5 [](/posts/we-stopped-using-serverless-the-results-are-insane--lq0hyyx5x "We stopped using serverless. The results are insane.") Video Theo - t3․gg · 2y We stopped using serverless. The results are insane. Upload Thing V7 drastically improves file upload speeds, making uploads five times faster. Major changes include seamless pausing and resuming of uploads, integration support for Remix, and the transition from relying on direct S3 uploads to a custom upload server, simplifying the process and enhancing control. This update reduces the number of steps and requests needed, enhances security, and unlocks new features like bring-your-own-bucket and better file validation. 52 3 6. 6 [](/posts/how-to-upload-to-s3-in-next-js-and-save-references-in-postgres-6q6tkowau "How to upload to S3 in Next.js and save references in Postgres") Article Community Picks · 2y How to upload to S3 in Next.js and save references in Postgres Learn how to add a feature to a Next.js app that allows users to upload files to Amazon S3 using presigned URLs, and save references to these files in a Postgres database. The guide covers creating a Neon project, setting up an S3 bucket, creating IAM user access keys, and setting up a Next.js application with necessary API routes to handle file uploads and database operations. 50 1 7. 7 [](/posts/how-an-empty-s3-bucket-can-make-your-aws-bill-explode-xptawo693 "How an empty S3 bucket can make your AWS bill explode") Article Lobsters · 2y How an empty S3 bucket can make your AWS bill explode Learn how an empty S3 bucket can result in a skyrocketing AWS bill due to unauthorized requests. Find out why a popular open-source tool caused the influx of requests and how you can enhance the security of your S3 bucket. 39 8. 8 [](/posts/s3-is-the-new-sftp-ilyh9akoc "S3 Is the New SFTP") Article Materialized View · 2y S3 Is the New SFTP Fintech companies handle diverse data processing tasks, including shuffling files between vendors and partners, often using SFTP. Transitioning to modern data lakehouses using S3, Apache Iceberg, and Apache Parquet can centralize and streamline this process. This new method allows ease of access and management while maintaining advantages such as fast transfers and central access control. Although challenges like schema evolution remain, adopting data lakehouses can benefit companies seeking efficient and scalable data solutions. The trend is supported by rising customer demand and the involvement of startups providing innovative data export platforms. 38 2 9. 9 [](/posts/building-a-distributed-log-using-s3-under-150-lines-of-go--pdebnqeon "Building a distributed log using S3 (under 150 lines of Go)") Article Lobsters · 2y Building a distributed log using S3 (under 150 lines of Go) The post describes how to implement a durable, distributed, and highly available log using AWS S3 in less than 150 lines of Go. Key highlights include the structure of the log interface, the implementation of the Append and Read operations, handling concurrent writes with S3 conditional writes, and failover/crash recovery mechanisms. The open-source project includes code and tests, with several open issues for further improvements. 36 10. 10 [](/posts/different-approaches-to-reduce-aws-s3-file-upload-time-using-aws-sdk-v3-in-nodejs--fntdbx52x "Different approaches to reduce AWS S3 file upload time using AWS-SDK v3 in NodeJS.") Article DEV · 2y Different approaches to reduce AWS S3 file upload time using AWS-SDK v3 in NodeJS. Learn different approaches to reduce AWS S3 file upload time using AWS-SDK v3 in NodeJS. Includes buffered upload, multipart upload, and file upload as a stream. 35 11. 11 [](/posts/aws-s3-deep-dive-jvlib9lus "AWS S3 Deep Dive") Article Medium · 2y AWS S3 Deep Dive Amazon S3 (Simple Storage Service) is a scalable object storage service offering high availability, security, and performance for various use cases like data lakes, backups, and big data analytics. It distinguishes itself from block storage and file storage, providing a flat storage structure through objects and buckets. Key features include data encryption, access control via IAM policies, and different bucket types optimized for various tasks such as general purpose, directory-specific, or table-based analytics. S3 also enhances data integrity and upload management via multipart uploads, versioning, and checksum validation. Additional services include S3 Object Lambda for real-time data transformation and S3 Batch Operations for large-scale data management tasks. 33 12. 12 [](/posts/how-we-decreased-our-monthly-aws-costs-from-10-000-to-1-500-6o6qc3vxs "How We Decreased Our Monthly AWS Costs from $10,000 to $1,500") Article Medium · 2y How We Decreased Our Monthly AWS Costs from $10,000 to $1,500 Managing cloud infrastructure can become costly if not optimized. This guide shares steps to significantly reduce AWS costs from $10,000 to $1,500 per month. Key actions include cleaning up ECR images, optimizing ECS Fargate services, managing S3 storage, converting VPN usage to SSH tunnels, right-sizing RDS and DocumentDB instances, purchasing ECS savings plans, consolidating load balancers, removing redundant IPv4 addresses and environments, and right-sizing remaining resources. 33 1 13. 13 [](/posts/private-empty-s3-bucket-cost-me-1300-kwmzkjjms "Private EMPTY S3 Bucket COST ME $1300") Video ThePrimeTime · 2y Private EMPTY S3 Bucket COST ME $1300 Discover how an empty private S3 bucket can cause a skyrocketing AWS bill, due to unauthorized requests from third parties and misconfigurations of an open-source tool. Learn about the potential risks and security measures to prevent such incidents. 33 1 14. 14 [](/posts/node-js-createreadstream-and-aws-multipartupload-unlocking-memory-and-network-efficient-file-upload-pnc0ghtbh "Node.js CreateReadStream and AWS MultipartUpload: Unlocking Memory and Network Efficient File Uploading Strategy.") Article AWS Tip · 2y Node.js CreateReadStream and AWS MultipartUpload: Unlocking Memory and Network Efficient File Uploading Strategy. Learn how to optimize file uploading in Node.js using fs.createReadStream and AWS MultipartUpload. Avoid high memory usage and improve performance when dealing with large files. AWS recommends using MultipartUpload for files over 100 MB. 24 2 15. 15 [](/posts/the-last-amazon-s3-video-you-ll-need-to-watch-presigned-urls-multipart-upload--ouwu4doiw "The Last Amazon S3 Video You'll Need To Watch (Presigned URLs, Multipart Upload)") Video Community Picks · 2y The Last Amazon S3 Video You'll Need To Watch (Presigned URLs, Multipart Upload) Learn best practices for working with Amazon S3 from a .NET application. This guide covers using the S3 SDK, working with pre-signed URLs, and handling multi-part uploads for large files. Practical tips include navigating the AWS console, configuring S3 buckets, and implementing efficient file upload and download functionalities. Explore examples on configuring CORS and using client-side applications for direct file interactions with S3. 22 16. 16 [](/posts/how-to-create-an-aws-s3-bucket-for-postgres-backups-5fkasv3ev "How to create an AWS S3 bucket for Postgres backups") Article Neon · 2y How to create an AWS S3 bucket for Postgres backups Learn how to set up an AWS S3 bucket to store Postgres backups, a useful step for managing backups in a multi-tenant architecture. This guide includes setting up AWS Providers and Roles, creating an OIDC Identity Provider and Role, and configuring an S3 bucket with an updated policy. The process involves using GitHub Actions to automate transfers, ideal for handling numerous projects efficiently. 21 17. 17 [](/posts/how-to-upload-large-files-efficiently-with-aws-s3-multipart-upload-iejgugtso "​​How to Upload Large Files Efficiently with AWS S3 Multipart Upload") Article freeCodeCamp · 2y ​​How to Upload Large Files Efficiently with AWS S3 Multipart Upload Efficiently handle large file uploads using Amazon S3 multipart upload. This approach splits a large file into smaller parts, uploads them independently, and combines them to create the final object. The guide covers setting up AWS S3, configuring a backend with Node.js, and building a React frontend for the upload process. This method enhances upload performance, adds fault tolerance, and allows pausing and resuming of uploads, making it suitable for large files and unstable networks. 21 18. 18 [](/posts/how-to-use-time-to-live-in-event-driven-architecture-in-aws-qbefuvaxb "How to Use Time To Live in Event-Driven Architecture in AWS") Article freeCodeCamp · 2y How to Use Time To Live in Event-Driven Architecture in AWS Time to Live (TTL) is a concept used to determine the lifespan of data in a distributed system. It is beneficial for ephemeral data like cached content, analytic metrics, and short-lived social media content. However, it is less useful for permanent data like media for streaming platforms or bank transactions. AWS services like SQS, S3, and DynamoDB support TTL, allowing developers to automate data expiration and efficiently manage resources. TTL can also be integrated with event-driven architectures to execute actions upon data expiration. 20 1 19. 19 [](/posts/1-5-million-pdfs-in-25-minutes-vzmw8tegz "1.5+ million PDFs in 25 minutes") Article Lobsters · 3y 1.5+ million PDFs in 25 minutes Zerodha shares their journey of rethinking the architecture and building an architecture from scratch to process, generate, digitally sign, and email 1.5+ million PDF contract notes in about 25 minutes. 20 20. 20 [](/posts/s3-is-a-security-nightmare-common-exploit-showcase--clxezgkcv "S3 Is A Security Nightmare (Common Exploit Showcase)") Video Theo - t3․gg · 2y S3 Is A Security Nightmare (Common Exploit Showcase) S3 has security issues and common vulnerabilities in uploads that can lead to cross-site scripting and unauthorized file overrides. Several third-party libraries exist to simplify S3 usage. 18 1 21. 21 [](/posts/how-to-secure-an-s3-bucket-on-aws--powrjlkaj "How to secure an S3 bucket on AWS?") Article Snyk · 2y How to secure an S3 bucket on AWS? Amazon S3 provides scalable, high-performance cloud storage, but securing S3 buckets is crucial to prevent data breaches. Key vulnerabilities include misconfigured permissions, unencrypted data, and lack of monitoring. Best practices for S3 bucket security include safeguarding permissions using IAM roles, enforcing encryption, and implementing secure transport policies like HTTPS. Regular auditing and using tools like Snyk for automated detection and remediation of cloud misconfigurations can significantly bolster security. 16 22. 22 [](/posts/gitops-project-todo-application-using-circle-ci-and-argo-cd--zknzcb3rz "GitOps Project — ToDo Application, using circle ci and Argo CD.") Article AWS Tip · 3y GitOps Project — ToDo Application, using circle ci and Argo CD. This post provides instructions on how to create an s3 bucket for backend storage, a DynamoDB to lock the state file, and how to set up Circle CI and Argo CD for a GitOps project to deploy a ToDo application. 16 23. 23 [](/posts/building-infrastructure-to-serve-millions-of-users-on-aws-preparation-yo6sauacz "Building Infrastructure to Serve Millions of Users on AWS: Preparation") Article Faun · 2y Building Infrastructure to Serve Millions of Users on AWS: Preparation This post provides step-by-step instructions on building infrastructure to serve millions of users on AWS. It covers topics such as creating virtual machines, installing WordPress on a virtual machine, creating a database on AWS, and connecting WordPress to an S3 bucket. 15 24. 24 [](/posts/aws-s3-and-endpoints-benefits-bwr7rdddk "AWS S3 and Endpoints Benefits") Article AWS in Plain English · 3y AWS S3 and Endpoints Benefits AWS S3 is a highly available object storage service with security and encryption features. There are different ways to access S3 bucket, including using VPC Gateway Endpoint or VPC Interface Endpoint. Consider using interface endpoint for secure access across regions or from on-premises. 15 25. 25 [](/posts/hacking-misconfigured-aws-s3-buckets-a-complete-guide-9baxzwxhc "Hacking misconfigured AWS S3 buckets: A complete guide") Article Hacker News · 2y Hacking misconfigured AWS S3 buckets: A complete guide AWS S3 buckets are widely used for storing both public and sensitive data, but misconfigurations can lead to serious security issues. The guide covers common security misconfigurations in AWS S3, ways to enumerate and test S3 buckets for vulnerabilities, and useful tools for automating these checks. It also provides practical commands and tips for validating S3 bucket permissions, ensuring access controls, and maintaining security practices such as S3 versioning. 14 [See all AWS S3 archives](/tags/aws-s3/best-of) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@graph":[{"@type":"CollectionPage","@id":"https://daily.dev/tags/aws-s3/best-of/2024#page","url":"https://daily.dev/tags/aws-s3/best-of/2024","name":"Best AWS S3 Posts of 2024","description":"The most upvoted AWS S3 posts of 2024, curated by the daily.dev community.","isPartOf":{"@type":"WebSite","url":"https://daily.dev"}},{"@type":"ItemList","@id":"https://daily.dev/tags/aws-s3/best-of/2024#items","numberOfItems":25,"itemListElement":[{"@type":"ListItem","position":1,"url":"https://daily.dev/posts/how-i-created-an-aws-system-to-crush-the-architect-exam-uqvrx0aio","name":"How I created an AWS System to crush the Architect exam"},{"@type":"ListItem","position":2,"url":"https://daily.dev/posts/cloudlena-s3manager-a-web-gui-for-your-s3-buckets-kb727nram","name":"cloudlena/s3manager: A Web GUI for your S3 buckets"},{"@type":"ListItem","position":3,"url":"https://daily.dev/posts/s3-is-showing-its-age-cnip6rya8","name":"S3 Is Showing Its Age"},{"@type":"ListItem","position":4,"url":"https://daily.dev/posts/i-wish-i-knew-this-one-thing-before-learning-aws-jqjhmvy6n","name":"I Wish I Knew This One Thing Before Learning AWS"},{"@type":"ListItem","position":5,"url":"https://daily.dev/posts/we-stopped-using-serverless-the-results-are-insane--lq0hyyx5x","name":"We stopped using serverless. The results are insane."},{"@type":"ListItem","position":6,"url":"https://daily.dev/posts/how-to-upload-to-s3-in-next-js-and-save-references-in-postgres-6q6tkowau","name":"How to upload to S3 in Next.js and save references in Postgres"},{"@type":"ListItem","position":7,"url":"https://daily.dev/posts/how-an-empty-s3-bucket-can-make-your-aws-bill-explode-xptawo693","name":"How an empty S3 bucket can make your AWS bill explode"},{"@type":"ListItem","position":8,"url":"https://daily.dev/posts/s3-is-the-new-sftp-ilyh9akoc","name":"S3 Is the New SFTP"},{"@type":"ListItem","position":9,"url":"https://daily.dev/posts/building-a-distributed-log-using-s3-under-150-lines-of-go--pdebnqeon","name":"Building a distributed log using S3 (under 150 lines of Go)"},{"@type":"ListItem","position":10,"url":"https://daily.dev/posts/different-approaches-to-reduce-aws-s3-file-upload-time-using-aws-sdk-v3-in-nodejs--fntdbx52x","name":"Different approaches to reduce AWS S3 file upload time using AWS-SDK v3 in NodeJS."},{"@type":"ListItem","position":11,"url":"https://daily.dev/posts/aws-s3-deep-dive-jvlib9lus","name":"AWS S3 Deep Dive"},{"@type":"ListItem","position":12,"url":"https://daily.dev/posts/how-we-decreased-our-monthly-aws-costs-from-10-000-to-1-500-6o6qc3vxs","name":"How We Decreased Our Monthly AWS Costs from $10,000 to $1,500"},{"@type":"ListItem","position":13,"url":"https://daily.dev/posts/private-empty-s3-bucket-cost-me-1300-kwmzkjjms","name":"Private EMPTY S3 Bucket COST ME $1300"},{"@type":"ListItem","position":14,"url":"https://daily.dev/posts/node-js-createreadstream-and-aws-multipartupload-unlocking-memory-and-network-efficient-file-upload-pnc0ghtbh","name":"Node.js CreateReadStream and AWS MultipartUpload: Unlocking Memory and Network Efficient File Uploading Strategy."},{"@type":"ListItem","position":15,"url":"https://daily.dev/posts/the-last-amazon-s3-video-you-ll-need-to-watch-presigned-urls-multipart-upload--ouwu4doiw","name":"The Last Amazon S3 Video You'll Need To Watch (Presigned URLs, Multipart Upload)"},{"@type":"ListItem","position":16,"url":"https://daily.dev/posts/how-to-create-an-aws-s3-bucket-for-postgres-backups-5fkasv3ev","name":"How to create an AWS S3 bucket for Postgres backups"},{"@type":"ListItem","position":17,"url":"https://daily.dev/posts/how-to-upload-large-files-efficiently-with-aws-s3-multipart-upload-iejgugtso","name":"​​How to Upload Large Files Efficiently with AWS S3 Multipart Upload"},{"@type":"ListItem","position":18,"url":"https://daily.dev/posts/how-to-use-time-to-live-in-event-driven-architecture-in-aws-qbefuvaxb","name":"How to Use Time To Live in Event-Driven Architecture in AWS"},{"@type":"ListItem","position":19,"url":"https://daily.dev/posts/1-5-million-pdfs-in-25-minutes-vzmw8tegz","name":"1.5+ million PDFs in 25 minutes"},{"@type":"ListItem","position":20,"url":"https://daily.dev/posts/s3-is-a-security-nightmare-common-exploit-showcase--clxezgkcv","name":"S3 Is A Security Nightmare (Common Exploit Showcase)"},{"@type":"ListItem","position":21,"url":"https://daily.dev/posts/how-to-secure-an-s3-bucket-on-aws--powrjlkaj","name":"How to secure an S3 bucket on AWS?"},{"@type":"ListItem","position":22,"url":"https://daily.dev/posts/gitops-project-todo-application-using-circle-ci-and-argo-cd--zknzcb3rz","name":"GitOps Project — ToDo Application, using circle ci and Argo CD."},{"@type":"ListItem","position":23,"url":"https://daily.dev/posts/building-infrastructure-to-serve-millions-of-users-on-aws-preparation-yo6sauacz","name":"Building Infrastructure to Serve Millions of Users on AWS: Preparation"},{"@type":"ListItem","position":24,"url":"https://daily.dev/posts/aws-s3-and-endpoints-benefits-bwr7rdddk","name":"AWS S3 and Endpoints Benefits"},{"@type":"ListItem","position":25,"url":"https://daily.dev/posts/hacking-misconfigured-aws-s3-buckets-a-complete-guide-9baxzwxhc","name":"Hacking misconfigured AWS S3 buckets: A complete guide"}]},{"@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Tags","item":"https://daily.dev/tags"},{"@type":"ListItem","position":3,"name":"AWS S3","item":"https://daily.dev/tags/aws-s3"},{"@type":"ListItem","position":4,"name":"Best of","item":"https://daily.dev/tags/aws-s3/best-of"},{"@type":"ListItem","position":5,"name":"2024"}]}]} ```