--- title: They All Write, Store, and Read Data: How to Pick The... description: A conceptual guide walks through the landscape of database types—SQLite, Postgres, DuckDB/OLAP, graph, vector, document, key/value, and time-series—framing the... canonical: https://daily.dev/posts/they-all-write-store-and-read-data-how-to-pick-the-right-database-qc0uzzyek twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: They All Write, Store, and Read Data: How to Pick The Right Database | daily.dev og:description: A conceptual guide walks through the landscape of database types—SQLite, Postgres, DuckDB/OLAP, graph, vector, document, key/value, and time-series—framing the... og:url: https://daily.dev/posts/they-all-write-store-and-read-data-how-to-pick-the-right-database-qc0uzzyek og:image: https://api.daily.dev/og/posts/Qc0UZZYeK.png og:image:alt: They All Write, Store, and Read Data: How to Pick The Right Database og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # They All Write, Store, and Read Data: How to Pick The Right Database **[MotherDuck](https://daily.dev/sources/motherduck)** · 16 min read · 11 upvotes · 2 comments ## Summary A conceptual guide walks through the landscape of database types—SQLite, Postgres, DuckDB/OLAP, graph, vector, document, key/value, and time-series—framing the choice around the kind of question you ask most often rather than which database is 'best'. It explains why SQLite works until concurrency appears, why Postgres suits ~80% of use cases, and dives deeper into why DuckDB (and MotherDuck) is fast for analytics: columnar storage, vectorized execution, predicate pushdown, zone maps, compression (zstd vs gzip vs brotli), embedded/in-process execution, and query optimizer rewrite rules like filter reordering and statistics propagation. It closes with quick capsule comparisons of Neo4j, vector databases (LanceDB, Qdrant, pgvector), MongoDB, Redis/DynamoDB, and time-series/geospatial databases, plus a plug for MotherDuck and DBngin. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Why is zstd usually the default compression codec for Parquet files instead of gzip? Zstd strikes the best balance between compression ratio and speed: it achieves roughly 3-5x compression with decompression speeds of about 1-2 GB/s, compared to gzip's 3-4x ratio but much slower 0.3-0.5 GB/s decompression. Since decompression speed matters most when reading over a network or object storage, gzip's slowness makes it a poor choice, while zstd remains fast enough to avoid becoming the bottleneck. _When deciding on file compression for analytics pipelines, daily.dev surfaces practical breakdowns like this for engineers weighing storage tradeoffs._ ### What is predicate pushdown and how does it speed up queries on columnar databases like DuckDB? Predicate pushdown pushes a query's filter as close to the data scan as possible, letting the scan itself decide which rows to return rather than filtering after loading everything. Combined with zone maps that store min/max metadata per row group, a filter like price >= 1500 can skip decompressing entire row groups whose maximum value falls below the threshold, avoiding unnecessary reads especially over slower storage like S3. _Engineers optimizing slow analytical queries can find deep dives like this on daily.dev when comparing columnar database internals._ ### When should I use Postgres versus a specialized database like DuckDB, MongoDB, or Redis? Postgres handles roughly 80% of use cases well and is the right default to start with, being good at writing, storing, and reading data when multiple users share it. Switch to DuckDB or an OLAP engine for heavy aggregation over years of data, MongoDB when documents need flexible nested shapes, Redis for ultra-fast known-key lookups, and vector or graph databases for similarity search or relationship queries respectively. _Developers deciding between database options can track comparisons like this on daily.dev before committing to an architecture._ ## Community discussion Top comments from developers on daily.dev. **@eusebioleite** · 0 upvotes > sqlite for everything **@anderspersson** · 0 upvotes > I never worked at a company where i can choose it have allways been a company rules. > > Both language and db. ## Similar posts on daily.dev - [How to Choose a Database for Your Next Project](https://daily.dev/posts/how-to-choose-a-database-for-your-next-project-cvd34xwxa) · BigData Boutique blog · 2 upvotes · 0 comments - [Why DuckDB is my first choice for data processing](https://daily.dev/posts/why-duckdb-is-my-first-choice-for-data-processing-sx7gglwxx) · Hacker News · 1 upvotes · 0 comments --- Tags: [#database](https://daily.dev/tags/database), [#postgresql](https://daily.dev/tags/postgresql), [#sqlite](https://daily.dev/tags/sqlite), [#duckdb](https://daily.dev/tags/duckdb) [View this post on daily.dev](https://daily.dev/posts/they-all-write-store-and-read-data-how-to-pick-the-right-database-qc0uzzyek) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"They All Write, Store, and Read Data: How to Pick The Right Database","url":"https://daily.dev/posts/they-all-write-store-and-read-data-how-to-pick-the-right-database-qc0uzzyek","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/they-all-write-store-and-read-data-how-to-pick-the-right-database-qc0uzzyek"},"datePublished":"2026-09-15T23:54:13.342Z","dateModified":"2026-09-15T23:54:39.770Z","description":"A conceptual guide walks through the landscape of database types—SQLite, Postgres, DuckDB/OLAP, graph, vector, document, key/value, and time-series—framing the...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/7c0eb838be20eb51cf28764db8c46211?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/7c0eb838be20eb51cf28764db8c46211?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"MotherDuck","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"MotherDuck","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/f084d6835eef4c2b83f889f9a4e85ed6","url":"https://daily.dev/sources/motherduck"},"commentCount":2,"discussionUrl":"https://daily.dev/posts/they-all-write-store-and-read-data-how-to-pick-the-right-database-qc0uzzyek","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":11},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":2}],"keywords":"database,postgresql,sqlite,duckdb","timeRequired":"PT16M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"MotherDuck","item":"https://daily.dev/sources/motherduck"},{"@type":"ListItem","position":3,"name":"They All Write, Store, and Read Data: How to Pick The Right Database"}]} {"@context":"https://schema.org","@type":"WebPage","@id":"https://daily.dev/posts/they-all-write-store-and-read-data-how-to-pick-the-right-database-qc0uzzyek","comment":[{"@type":"Comment","text":"sqlite for everything","datePublished":"2026-09-17T01:55:00.852Z","url":"https://daily.dev/posts/Qc0UZZYeK#c-6PcEyo9Jn","author":{"@type":"Person","name":"Eusébio Leite","url":"https://daily.dev/eusebioleite","image":"https://media.daily.dev/image/upload/s--ccQB46fi--/f_auto/v1789741448/avatars/avatar_VL9384KKhHkUpz2NoKYmL?_a=BAMAMicg0"}},{"@type":"Comment","text":"I never worked at a company where i can choose it have allways been a company rules.\nBoth language and db.","datePublished":"2026-09-17T17:44:52.474Z","url":"https://daily.dev/posts/Qc0UZZYeK#c-IOGRuDZnL","author":{"@type":"Person","name":"Anders Persson","url":"https://daily.dev/anderspersson","image":"https://avatars.githubusercontent.com/u/1372776?v=4"}}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/they-all-write-store-and-read-data-how-to-pick-the-right-database-qc0uzzyek#faq","mainEntity":[{"@type":"Question","name":"Why is zstd usually the default compression codec for Parquet files instead of gzip?","acceptedAnswer":{"@type":"Answer","text":"Zstd strikes the best balance between compression ratio and speed: it achieves roughly 3-5x compression with decompression speeds of about 1-2 GB/s, compared to gzip's 3-4x ratio but much slower 0.3-0.5 GB/s decompression. Since decompression speed matters most when reading over a network or object storage, gzip's slowness makes it a poor choice, while zstd remains fast enough to avoid becoming the bottleneck. When deciding on file compression for analytics pipelines, daily.dev surfaces practical breakdowns like this for engineers weighing storage tradeoffs."}},{"@type":"Question","name":"What is predicate pushdown and how does it speed up queries on columnar databases like DuckDB?","acceptedAnswer":{"@type":"Answer","text":"Predicate pushdown pushes a query's filter as close to the data scan as possible, letting the scan itself decide which rows to return rather than filtering after loading everything. Combined with zone maps that store min/max metadata per row group, a filter like price >= 1500 can skip decompressing entire row groups whose maximum value falls below the threshold, avoiding unnecessary reads especially over slower storage like S3. Engineers optimizing slow analytical queries can find deep dives like this on daily.dev when comparing columnar database internals."}},{"@type":"Question","name":"When should I use Postgres versus a specialized database like DuckDB, MongoDB, or Redis?","acceptedAnswer":{"@type":"Answer","text":"Postgres handles roughly 80% of use cases well and is the right default to start with, being good at writing, storing, and reading data when multiple users share it. Switch to DuckDB or an OLAP engine for heavy aggregation over years of data, MongoDB when documents need flexible nested shapes, Redis for ultra-fast known-key lookups, and vector or graph databases for similarity search or relationship queries respectively. Developers deciding between database options can track comparisons like this on daily.dev before committing to an architecture."}}]} ```