Hacker News: adastral

I've operated petabyte-scale ClickHouse clusters for 5 years

adastral — Mon, 07 Sep 2026 18:05:04 +0000

Article URL: https://www.tinybird.co/blog/what-i-learned-operating-clickhouse

Comments URL: https://news.ycombinator.com/item?id=49601138

Points: 191

# Comments: 73

New comment by adastral in "Massively Parallel Postgres Backups"

adastral — Mon, 03 Aug 2026 22:21:32 +0000

How do you cope with distributed transactions? (I'm not sure if you support them?)

I know that Citus has a `citus_create_restore_point()` (or so) function that, when called, guarantees that no 2PC commits are in flight and creates a WAL restore point in every shard. Therefore, restoring shards to that point will leave the DB in a consistent state. Do you do something similar?

Pg_hardstorage: PostgreSQL Backup, Done Right

adastral — Tue, 23 Jun 2026 13:13:15 +0000

Article URL: https://github.com/cybertec-postgresql/pg_hardstorage

Comments URL: https://news.ycombinator.com/item?id=48644482

Points: 5

# Comments: 0

RawTree

adastral — Thu, 11 Jun 2026 19:00:30 +0000

Article URL: https://rawtree.com/blog/introducing-rawtree

Comments URL: https://news.ycombinator.com/item?id=48494890

Points: 3

# Comments: 0

We moved our growth analytics back into Tinybird

adastral — Sun, 07 Jun 2026 15:14:00 +0000

Article URL: https://www.tinybird.co/blog/why-we-moved-our-growth-analytics-back-into-tinybird

Comments URL: https://news.ycombinator.com/item?id=48435628

Points: 3

# Comments: 0

Key Joins (SQL Proposal)

adastral — Fri, 29 May 2026 08:11:49 +0000

Article URL: https://keyjoin.org/

Comments URL: https://news.ycombinator.com/item?id=48320456

Points: 2

# Comments: 0

New comment by adastral in "A staff engineer's journey with Claude Code"

adastral — Wed, 03 Sep 2025 14:53:05 +0000

PostgresTV livestreams "vibe coding" 1h sessions implementing small PostgreSQL features with Cursor (mostly claude-4-sonnet model) every week, by experienced PostgreSQL contributors. [0] is their latest stream.

I personally have not watched much, but it sounds just like what you are looking for!

[0] https://www.youtube.com/watch?v=3MleDtXZUlM

Git-worktree – Manage multiple working trees

adastral — Wed, 20 Aug 2025 14:21:17 +0000

Article URL: https://git-scm.com/docs/git-worktree

Comments URL: https://news.ycombinator.com/item?id=44962250

Points: 8

# Comments: 0

Benchmarking Is Hard, Sometimes

adastral — Thu, 05 Jun 2025 11:35:31 +0000

Article URL: https://vondra.me/posts/benchmarking-is-hard-sometimes/

Comments URL: https://news.ycombinator.com/item?id=44190582

Points: 2

# Comments: 0

Decomposing Transactional Systems

adastral — Thu, 17 Apr 2025 22:00:30 +0000

Article URL: https://transactional.blog/blog/2025-decomposing-transactional-systems

Comments URL: https://news.ycombinator.com/item?id=43722692

Points: 18

# Comments: 0

Dynamically estimating and scaling Postgres' working set size (2024)

adastral — Thu, 06 Mar 2025 11:02:27 +0000

Article URL: https://neon.tech/blog/dynamically-estimating-and-scaling-postgres-working-set-size

Comments URL: https://news.ycombinator.com/item?id=43278726

Points: 2

# Comments: 0

New comment by adastral in "Curl-Impersonate"

adastral — Mon, 30 Dec 2024 13:57:26 +0000

> I default to a headless browser

Headless browsers consume orders of magnitude more resources, and execute far more requests (e.g. fetching images) than a common webscraping job would require. Having run webscraping at scale myself, the cost of operating headless browsers made us only use them as a last resort.

AWS CloudWatch and AWS OpenSearch launch an integrated analytics experience

adastral — Mon, 02 Dec 2024 14:57:06 +0000

Article URL: https://aws.amazon.com/blogs/aws/new-amazon-cloudwatch-and-amazon-opensearch-service-launch-an-integrated-analytics-experience/

Comments URL: https://news.ycombinator.com/item?id=42296796

Points: 2

# Comments: 0

New comment by adastral in "Starcraft (A History in Two Acts)"

adastral — Sun, 07 Jul 2024 08:56:57 +0000

From https://www.codeofhonor.com/blog/the-starcraft-path-finding-...

> whenever harvesters are on their way to get minerals, or when they’re on the way back carrying those minerals, they ignore collisions with other units

Maybe that's part of the fluid behaviour you noticed.

The blog contains more technical posts about StarCraft 2, so you might find that "special sauce" somewhere else in there :)

New comment by adastral in "The Great Migration from MongoDB to PostgreSQL"

adastral — Fri, 29 Mar 2024 08:56:27 +0000

Not sure if this is what the above comment means by "atomic", but a shortcoming of Postgres' JSON support is that it will have to rewrite an entire JSON object every time a part of it gets updated, no matter how many keys the update really affected. E.g. if I update an integer in a 100MB JSON object, Postgres will write ~100MB (plus WAL, TOAST overhead, etc.), not just a few bytes. I imagine this can be a no-go for certain use cases.

New comment by adastral in "How Figma's databases team lived to tell the scale"

adastral — Thu, 14 Mar 2024 18:49:28 +0000

Around ten heavily-updated (50-400k updated rows/min) tables ranging between 500M and 5B rows, with a couple tables over 40B rows each (5TB each IIRC).

New comment by adastral in "How Figma's databases team lived to tell the scale"

adastral — Thu, 14 Mar 2024 18:38:29 +0000

I see they don't mention Citus (https://github.com/citusdata/citus), which is already a fairly mature native Postgres extension. From the details given in the article, it sounds like they just reimplemented it.

I wonder if they were unaware of it or disregarded it for a reason —I currently am in a similar situation as the one described in the blog, trying to shard a massive Postgres DB.

New comment by adastral in "Autogenerating a Book Series from Three Years of iMessages"

adastral — Thu, 07 Mar 2024 19:23:50 +0000

> they'll be encrypted

Since some months (years?) ago, WhatsApp lets you set up your own encryption password for the DB backup. I set one up and used https://github.com/ElDavoo/wa-crypt-tools to get access to the decrypted SQLite and run some analytics over my messages :)