---
title: "DuckDB solved the query engine. The workflow around it is still messy"
url: https://daily.dev/posts/duckdb-solved-the-query-engine-the-workflow-around-it-is-still-messy-8ctaacm1l
source_url: https://daily.dev/posts/duckdb-solved-the-query-engine-the-workflow-around-it-is-still-messy-8ctaacm1l
type: freeform
source: "Database Daily"
author: "Dmitry Narizhnykh"
published: 2026-05-13T12:15:05.518Z
updated: 2026-05-13T12:15:22.252Z
tags: ["python", "automation", "big-data", "data-engineering", "duckdb"]
reading_time: 1
upvotes: 1
comments: 0
language: en
---

> ## Documentation Index
> Fetch the complete documentation index at: https://daily.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# DuckDB solved the query engine. The workflow around it is still messy

**[Database Daily](https://daily.dev/sources/databasedaily)** · [@slotix](https://daily.dev/slotix) · 1 min read · 1 upvotes · 0 comments

## Summary

DuckDB has largely solved the query engine problem, making cross-source queries across Postgres and Parquet files in S3 straightforward. The real challenge lies in the repeatable workflow around those queries: managing saved connections, S3 paths, schema checks, aliases, result exports, and reruns. This glue logic often ends up scattered across Python scripts, Makefiles, or notebooks. The post introduces a structured workspace layer built around DuckDB in DBConvert Streams to address this gap.

## Content

DuckDB makes it pretty easy to query across Postgres and Parquet files in S3.

The query itself is not the hard part anymore.

The mess starts when the same query needs to be reused:

- saved DB connections
- S3 paths
- schema checks
- aliases
- result exports
- reruns
- sometimes using the result as input for load or migration

That is the part people often hide in a Python script, Makefile, notebook, or some “temporary” glue code that quietly becomes important.

The article came from the workflow we built around DuckDB in DBConvert Streams.

DuckDB is the engine.

The missing layer is often the repeatable workspace around it.

https://streams.dbconvert.com/blog/duckdb-cross-source-sql-workflow/

## Similar posts on daily.dev

- [Why DuckDB is my first choice for data processing](https://daily.dev/posts/why-duckdb-is-my-first-choice-for-data-processing-sx7gglwxx) · Hacker News · 1 upvotes · 0 comments
- [Run SQL queries on local parquet and delta files using DuckDB](https://daily.dev/posts/run-sql-queries-on-local-parquet-and-delta-files-using-duckdb-0qdy9r9y9) · The Art of Simplicity · 0 upvotes · 0 comments

---

Tags: [#python](https://daily.dev/tags/python), [#automation](https://daily.dev/tags/automation), [#big-data](https://daily.dev/tags/big-data), [#data-engineering](https://daily.dev/tags/data-engineering), [#duckdb](https://daily.dev/tags/duckdb)

[View this post on daily.dev](https://daily.dev/posts/duckdb-solved-the-query-engine-the-workflow-around-it-is-still-messy-8ctaacm1l)