Hacker News: roboben

New comment by roboben in "Moving our warehouse off DuckDB"

roboben — Tue, 03 Nov 2026 17:11:36 +0000

Its not Trino, its the catalog you use in the back.

New comment by roboben in "Moving our warehouse off DuckDB"

roboben — Mon, 02 Nov 2026 19:23:33 +0000

With DuckDB and the queries we got, we had to provision such huge pods/nodes that we made it work but could never get to an economical viability. In the end, most of our customers had pods running 46 vCPUs / 360GB and asked for more. DuckDB is amazing, it really does everything right on a single node to make your query run successfully to the end almost always but it just doesn't work in our cloud env. Imagine when a query is running for 12h with this since and then another 10 from the same customer come in :)

Trino on the other hand has a much more convenient distributed query concept, so you can much more dynamically assign resources.

New comment by roboben in "Moving our warehouse off DuckDB"

roboben — Mon, 02 Nov 2026 19:20:09 +0000

We tried Iceberg and it was too slow writing data is the short story.

>The fast OLAP stores just don't scale as well as going object store to a proper query engine like Trino

this exactly why we started with this project.

New comment by roboben in "Moving our warehouse off DuckDB"

roboben — Mon, 02 Nov 2026 19:12:41 +0000

Author here. Good question. AFAIK you cannot use Quack for anything that looks remotely like distributed querying.

New comment by roboben in "Moving our warehouse off DuckDB"

roboben — Mon, 02 Nov 2026 19:11:39 +0000

Author here. I agree. We spent most of the time to make it work at the scale of our large design partners and probably should have stepped back earlier. And yes, with an agent you can have a much larger scope hashed out faster than before.

New comment by roboben in "Moving our warehouse off DuckDB"

roboben — Mon, 02 Nov 2026 19:09:59 +0000

Author here. Our fork, like most of our work is public on Github: https://github.com/postHog/ducklake I think it does not include 100% of all changes since a few we upstreamed directly (to a few extensions related to Ducklake too) but if you are really interested, you can find them too.

New comment by roboben in "Moving our warehouse off DuckDB"

roboben — Mon, 02 Nov 2026 19:08:06 +0000

Author here. Good point. Trino is not made for fast response times. We are using it for long running data modelling jobs or nightly ETLs.

New comment by roboben in "Moving our warehouse off DuckDB"

roboben — Mon, 02 Nov 2026 19:06:45 +0000

Author here. We didn't leave Clickhouse. It still drives most of what Posthog offers. I missed that bit of context in my post I admit. Instead we are looking into a new stack we can give customers so they can run their warehousing workloads on it instead of the centralized CH.

Stateless CH on Iceberg sounds interesting, we looked into it but Delta/Iceberg had other downsides for us.

New comment by roboben in "Moving our warehouse off DuckDB"

roboben — Mon, 02 Nov 2026 19:03:30 +0000

Author here. This is not about our internal warehouse but the warehouse product we offer to customers. This is still running on our centralized Clickhouse. Long-running data modelling queries and multi-tenancy is a challenge on Clickhouse.

New comment by roboben in "Moving our warehouse off DuckDB"

roboben — Mon, 02 Nov 2026 19:00:11 +0000

Author here. We went past the POC stage to actually make real customer workloads work, which we did. Probably yes, we spent too much time not rethinking our approach but sunken cost fallacy so that's why.

New comment by roboben in "Moving our warehouse off DuckDB"

roboben — Mon, 02 Nov 2026 18:57:50 +0000

Author here. I admit I kinda left out the context which is important here. We did not replace Clickhouse. Instead we want to move our warehouse product from Clickhouse to another stack which suits the long modeling jobs and multi-tenancy better. We learned through all our early design partners running huge workloads what we need to change.

As much as I'd love to claim victory, we are not there yet :)

New comment by roboben in "Moving our warehouse off DuckDB"

roboben — Mon, 02 Nov 2026 18:55:00 +0000

[Author] Basically what we are doing with the help of Trino: a distributed query engine which can use as many resources as the query needs dynamically.

New comment by roboben in "Moving our warehouse off DuckDB"

roboben — Mon, 02 Nov 2026 18:54:00 +0000

[Author] Basically yes, but we spun up pods on EKS. It was hard for the customer to estimate how much they might need and hard for us to know how much capacity to provide. Given our plans on pricing on CPUHours (or similar) the logic was kinda broken.

New comment by roboben in "Moving our warehouse off DuckDB"

roboben — Mon, 02 Nov 2026 18:52:08 +0000

[Author] it was more like: we underestimated the problem and complexity of workloads our customers want to run. We did not rewrite anything but developing a new stack we can give to our customers doing warehousing. Agreed that this context is not really clear from the blog post.

New comment by roboben in "Moving our warehouse off DuckDB"

roboben — Mon, 02 Nov 2026 18:49:51 +0000

Hi! Author here.

We engineered, then we measured and then we tried to make it work, probably for a too long of a time before we decided to completely scrap our current architecture and go back to the drawing board. Happy to answer more questions.

New comment by roboben in "Spider venom kills varroa mites without harming honeybees"

roboben — Thu, 09 Jul 2026 13:35:34 +0000

mite popcorn with fresh honey sounds like a three star dish for $67

New comment by roboben in "Spider venom kills varroa mites without harming honeybees"

roboben — Thu, 09 Jul 2026 05:42:29 +0000

The hard truth these days is that the work of bee keeping is like 80% keeping the mites in check. Plus all current treatments render the honey inedible so you can only do it at the end of the season.

New comment by roboben in "How we run Firecracker VMs inside EC2 and start browsers in less than 1s"

roboben — Wed, 17 Jun 2026 18:50:51 +0000

docker is not a security boundary but a resource boundary.

New comment by roboben in "How we run Firecracker VMs inside EC2 and start browsers in less than 1s"

roboben — Wed, 17 Jun 2026 18:49:36 +0000

also i found them much less stable than metal instances running into weird kvm failures

New comment by roboben in "How we run Firecracker VMs inside EC2 and start browsers in less than 1s"

roboben — Wed, 17 Jun 2026 18:37:52 +0000

yes only c8i, m8i and r8i instance types support it. It is called nested virtualization[1]

[1] https://aws.amazon.com/about-aws/whats-new/2026/02/amazon-ec...