Super excited about Quack (partially due to the name). I use duckdb for both analytics and runtime, but I do have to serve/handle/manage a giant, multi-GiB duckdb file as effectively a runtime artifact[1]. I'm aware that this isn't the _perfect_ database for this, but the mix of it being fast, having spatial support, sane coding interfaces, great dbt integration, and me being able to do everything between "run a giant several hundred step dbt pipeline" to "query the output of said pipeline" to "read/query a csv on disk" with the exact same tool is just so nice. If I could centrally manage said asset more akin to a traditional database, I'd be very happy.
I've partially solved this with separate databases for different steps in the data pipeline(s) and have even experimented with Clickhouse as a complete alternative, but I really like way too many things about duckdb to replace it.
I built a platform for some midsize companies in a specific vertical that is basically a data warehouse with some LLM-driven dashboarding and query tools on top. Typical data size 5-150gb. So I built a service layer around duckdb, where each tenant gets their own duckdb. I'm also in the boat of knowing that duckdb is not the perfect solution for this (the classic use case is running it against local data on a laptop), but there's so much I like about it, and it's really nice that each tenant can have total separation with custom schemas and that it's straightforward to pile data into object storage and form your own lake.
And now with things like quack the sharp points around concurrency are relaxing, and it feels like the compromises of using it this way are disappearing.
I discovered DuckDB looking for a way to analyze Nginx access.log's and it's an amazing tool. I believe it should be a standard tool like ripgrep for devs.
DuckDB is one of the things I've been most excited about in a long time. Introduced it to projects at 3 companies since 2023, greatly lowering resource requirements and running it in a variety of environments. Just having the ability to do out of core bigger than memory data processing on lower end consumer grade hardware is remarkable.
Curious to learn more about how people are using it?
Are they downloading parquet files and running analyses locally, or are they connecting to Iceberg-like data lake and leveraging DuckDBs query engine capabilities or have you exposed an interface (REST, UI) to query your data?
- ETL pipelines running on K8s nodes. Using their streaming processing engine means I can run smaller pods/nodes if needed, for datasets that may have required large dataframe-like transformations that may have buffered a big dataset into memory previously.
- A CLI distributed to an internal team to do a postprocessing step on a large modeling dataset - to get it into a consumable format and upload it to a bucket as a .db file.
- A SvelteKit app that used the node duckdb bindings to attach to the .db on the bucket and explore the results through a suite of BI tools. These tables have millions of rows, and would be pretty heavy to store in PG. The DuckDB version works really, really well.
We use DuckDB WASM with parquet to build dashboards in-browser. It's cool to be able to write SQL directly in a browser and not have to rely on REST/Graphql/etc to access the data layer.
I have used it with WASM for some web applications for web use. I have also used with locally for querying 100 gigs of data. And I have used it in the cloud as the serverless gold layer for Apache superset.
We use WASM DuckDB as the target for an in-browser agentic feature. Generated SQL runs against the user's individual tables that then feed in-browser dashboards.
maybe a niche use case but i've found it's perfect to store & query random trivia/gameshow questions based on filters for my personal clones of things like Family Feud and Jeopardy
DuckDB out of the box may not be great. But you have DuckLake, Quack, and even DeepSeek made their own distributed DB based on DuckDB: https://github.com/deepseek-ai/smallpond
I don't think DuckDB itself can coordinate work across multiple nodes. But you could put it behind an HTTP layer and scale horizontally based on resource utilization?
Are there improvements in how memory_limit works? I often had DuckDB get OOM killed because it went beyond its limit. It's definitely one of the reasons why I usually have an AI tune the environment for my datasets.
I <3 DuckDB. It has become one of my go to tools for storing, data processing , integrations and now even graph. More importantly it's fun to use because it is so portable. Looking forward to v2.
I've been working on a demo database project, and have been really impressed by the UI. So glad they decided to put more effort into it, it has made building a "follow along" tutorial really nice.
The last year of DuckDB enhancements feel like the shift from in-process execution engine (which it is phenomenal at) to an engine that can serve as the foundation of a cloud data warehouse. I know the founders were reticent about not wanting to build that, but I have a feeling it is in the works.
If I could have a pet feature added to DuckDB, it would be some form of native ordered table. In a database like Clickhouse or any of the dedicated time series DBMSes or log stores, there’s a built-in concept that a table might have an order, and the database will optimize based on the order. But, for databases that are logically just bags of rows (traditional DBMSes and also DuckDB [0]), you either need an index or you need to rely on full table scans or at least scans of big blocks. DuckDB does the latter really well, but I think it would be quite nice for some workflows to have explicit ordering. Also, I bet compression could work a lot better with ordering hints.
All that being said, I’m quite excited about DuckDB 2.0. I want to give the improved VARIANT support a try.
[0] Documentation on DuckDB’s native format is rather sparse AFAICT. But the DDL has nothing resembling an ordered table.
Really looking forward to that new Async system, especially when reading/querying against thousands of parquet files. This is going to monumentally affect me and my work - I have to query against millions of massive parquet files and the speed has already been rather wonderful, but if those metrics are to be even 100% in range, this is going to make life so much better.
DuckDB is an in-process (now I guess less so with Quack) OLAP database for analytical workloads. PostgreSQL or MariaDB are OLTP row-oriented databases that are great for application/transaction-focused workloads but are less great when you want to query across a giant amount of data.
What do you plan on stripping and what's your target? The Emscripten based build is ~10Mb. I have a component build so I'd be interesting on how you'd like to break it up.
Looks like an awesome release, but the smell of AI from that post is horrid.
Here is a wild idea: is it really so hard to edit out sentences structured and punctuated like this - it's so painfully obvious and distracts from the content. The effect is real.
I don't really get an AI smell on this, in fact I see multiple parts that an AI would have corrected - grammatical issues, personal writing quirks, etc.
Writing similar to this: is quite a normal way of writing for technical articles - especially when you are trying to make the point clear and well organized.
I’m currently contemplating that MySQL apparently cannot do an INSTANT change of the collation of an unindexed column, even though, AFAICT, it has no effect whatsoever on the on-disk format or any data structure at all except for the metadata saying what the column type is.
I do not enjoy dealing with text encodings and collations in databases.
Disappointed, since I was expecting they would rewrite the implementation from C++ to Zig. I bet that would increase the number of positive pull requests they get, since most developers prefer to stay away from C++ nowadays.
I've partially solved this with separate databases for different steps in the data pipeline(s) and have even experimented with Clickhouse as a complete alternative, but I really like way too many things about duckdb to replace it.
[1]: If you care: https://skaldmaps.com/blog/2026/07/zip-codes-are-a-bad-spati...
And now with things like quack the sharp points around concurrency are relaxing, and it feels like the compromises of using it this way are disappearing.
I discovered DuckDB looking for a way to analyze Nginx access.log's and it's an amazing tool. I believe it should be a standard tool like ripgrep for devs.
Thanks to the team for everything!
Are they downloading parquet files and running analyses locally, or are they connecting to Iceberg-like data lake and leveraging DuckDBs query engine capabilities or have you exposed an interface (REST, UI) to query your data?
- ETL pipelines running on K8s nodes. Using their streaming processing engine means I can run smaller pods/nodes if needed, for datasets that may have required large dataframe-like transformations that may have buffered a big dataset into memory previously.
- A CLI distributed to an internal team to do a postprocessing step on a large modeling dataset - to get it into a consumable format and upload it to a bucket as a .db file.
- A SvelteKit app that used the node duckdb bindings to attach to the .db on the bucket and explore the results through a suite of BI tools. These tables have millions of rows, and would be pretty heavy to store in PG. The DuckDB version works really, really well.
I have used it with WASM for some web applications for web use. I have also used with locally for querying 100 gigs of data. And I have used it in the cloud as the serverless gold layer for Apache superset.
Excellent performance.
Does that make this account an alias as well?
"If we use this we get indefinite RAM indefinite CPU and do not need to host a server".
I had an impression that DuckDB was not great at distributing work to other machines, but good at doing it locally? Am I wrong?
I made a dry run extension a few months ago (https://github.com/aleda145/duckdb-dryrun), will be so nice to build it just once and know that it will always work.
Also urge anyone to make an extension, the template makes it quite smooth: https://github.com/duckdb/extension-template
> They decided to work together, and DuckLabs would become a co-founder of the new endeavor.
https://motherduck.com/about-us/
All that being said, I’m quite excited about DuckDB 2.0. I want to give the improved VARIANT support a try.
[0] Documentation on DuckDB’s native format is rather sparse AFAICT. But the DDL has nothing resembling an ordered table.
DuckDB is seriously an incredible utility.
What about the runtime size? I care this because I intend to run a stripped WASM version of DuckDB in browser.
Here is a wild idea: is it really so hard to edit out sentences structured and punctuated like this - it's so painfully obvious and distracts from the content. The effect is real.
Writing similar to this: is quite a normal way of writing for technical articles - especially when you are trying to make the point clear and well organized.
For me, this sentence pops out like a sore thumb
I do not enjoy dealing with text encodings and collations in databases.