--- title: Rust. AWS S3. Scripts. Python. Datetimes. Files. Etc. description: A developer describes hand-writing a Rust script to check an AWS S3 bucket full of tar and gz files instead of relying on an LLM-generated Python script. The... canonical: https://daily.dev/posts/rust-aws-s3-scripts-python-datetimes-files-etc--cngkavf9l twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Rust. AWS S3. Scripts. Python. Datetimes. Files. Etc. | daily.dev og:description: A developer describes hand-writing a Rust script to check an AWS S3 bucket full of tar and gz files instead of relying on an LLM-generated Python script. The... og:url: https://daily.dev/posts/rust-aws-s3-scripts-python-datetimes-files-etc--cngkavf9l og:image: https://api.daily.dev/og/posts/CNGkavf9L.png og:image:alt: Rust. AWS S3. Scripts. Python. Datetimes. Files. Etc. og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Rust. AWS S3. Scripts. Python. Datetimes. Files. Etc. **[Confessions of a Data Guy](https://daily.dev/sources/confessionsofadataguy)** · 8 min read · 0 upvotes · 0 comments ## Summary A developer describes hand-writing a Rust script to check an AWS S3 bucket full of tar and gz files instead of relying on an LLM-generated Python script. The S3 bucket held thousands of tar files, each containing maybe 10 gz flat files, with millions of files and tens of millions of records daily. The question was whether some individual .txt.gz files were missing, since Lambda unpacking jobs sometimes fail, usually from memory issues. An LLM could produce a Python script in 30 seconds, but it ran painfully slow. The author used the AWS Rust SDK, tokio async, and a tar crate, and found that writing Rust by hand felt good and reminded him why simplicity matters. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How do I list and filter S3 objects using the Rust AWS SDK with pagination? Use the aws-sdk-s3 crate's list_objects_v2 builder with bucket, prefix, and max_keys, then call into_paginator().send() to get a paginator you can iterate with while let Some(page_result) = paginator.next().await. Each page's contents() gives objects whose key() you can filter, for example checking if the key contains '.tar'. _Developers auditing S3 buckets in Rust can compare real SDK patterns like this through daily.dev._ ### How can I read the contents of a tar file stored in S3 without downloading it to disk in Rust? Fetch the object with the AWS SDK's get_object(), collect its body into bytes with response.body.collect().await.unwrap().into_bytes(), then wrap those bytes in a Cursor and pass it to tar::Archive::new to iterate entries in memory. This avoids writing the tar file to local disk before inspecting its contents. _Engineers handling tar archives in cloud storage can find working Rust patterns like this on daily.dev._ ## Similar posts on daily.dev - [Writing Rust … by Hand. No LLMs.](https://daily.dev/posts/writing-rust-by-hand-no-llms--zu6hsaxdj) · Confessions of a Data Guy · 23 upvotes · 8 comments - [AWS shows Rust love at re:Invent: 10 times faster than Kotlin, one tenth the latency of Go • DEVCLASS](https://daily.dev/posts/aws-shows-rust-love-at-re-invent-10-times-faster-than-kotlin-one-tenth-the-latency-of-go-devclas-2oyry2vcw) · DEVCLASS · 22 upvotes · 1 comments - [AWS Lambda Rust Support Reaches General Availability](https://daily.dev/posts/aws-lambda-rust-support-reaches-general-availability-vug2t65ta) · InfoQ · 2 upvotes · 0 comments --- Tags: [#python](https://daily.dev/tags/python), [#aws](https://daily.dev/tags/aws), [#rust](https://daily.dev/tags/rust), [#vibe-coding](https://daily.dev/tags/vibe-coding), [#aws-s3](https://daily.dev/tags/aws-s3) [View this post on daily.dev](https://daily.dev/posts/rust-aws-s3-scripts-python-datetimes-files-etc--cngkavf9l) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Rust. AWS S3. Scripts. Python. Datetimes. Files. Etc.","url":"https://daily.dev/posts/rust-aws-s3-scripts-python-datetimes-files-etc--cngkavf9l","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/rust-aws-s3-scripts-python-datetimes-files-etc--cngkavf9l"},"datePublished":"2026-10-28T20:36:37.865Z","dateModified":"2026-10-28T21:33:53.895Z","description":"A developer describes hand-writing a Rust script to check an AWS S3 bucket full of tar and gz files instead of relying on an LLM-generated Python script. The...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/6d918c5c937ccb2a0b444f25e0210dbc?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/6d918c5c937ccb2a0b444f25e0210dbc?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Confessions of a Data Guy","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Confessions of a Data Guy","logo":"https://media.daily.dev/image/upload/s--JEZMXhIv--/f_auto,q_auto/v1780213361/logos/confessionsofadataguy?_a=BAMAMiWQ0","url":"https://daily.dev/sources/confessionsofadataguy"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/rust-aws-s3-scripts-python-datetimes-files-etc--cngkavf9l","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"python,aws,rust,vibe-coding,aws-s3","timeRequired":"PT8M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Confessions of a Data Guy","item":"https://daily.dev/sources/confessionsofadataguy"},{"@type":"ListItem","position":3,"name":"Rust. AWS S3. Scripts. Python. Datetimes. Files. Etc."}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/rust-aws-s3-scripts-python-datetimes-files-etc--cngkavf9l#faq","mainEntity":[{"@type":"Question","name":"How do I list and filter S3 objects using the Rust AWS SDK with pagination?","acceptedAnswer":{"@type":"Answer","text":"Use the aws-sdk-s3 crate's list_objects_v2 builder with bucket, prefix, and max_keys, then call into_paginator().send() to get a paginator you can iterate with while let Some(page_result) = paginator.next().await. Each page's contents() gives objects whose key() you can filter, for example checking if the key contains '.tar'. Developers auditing S3 buckets in Rust can compare real SDK patterns like this through daily.dev."}},{"@type":"Question","name":"How can I read the contents of a tar file stored in S3 without downloading it to disk in Rust?","acceptedAnswer":{"@type":"Answer","text":"Fetch the object with the AWS SDK's get_object(), collect its body into bytes with response.body.collect().await.unwrap().into_bytes(), then wrap those bytes in a Cursor and pass it to tar::Archive::new to iterate entries in memory. This avoids writing the tar file to local disk before inspecting its contents. Engineers handling tar archives in cloud storage can find working Rust patterns like this on daily.dev."}}]} ```