Hacker News: vira28

New comment by vira28 in "Ask HN: What are you working on? (June 2026)"

vira28 — Mon, 15 Jun 2026 05:09:34 +0000

Building https://github.com/viggy28/streambed - a full Postgres running on S3 using DuckDB as the query engine.

New comment by vira28 in "Ask HN: Want to build something open source on nights and weekends together?"

vira28 — Sun, 14 Jun 2026 20:02:12 +0000

Certainly interested.

New comment by vira28 in "Ask HN: Want to build something open source on nights and weekends together?"

vira28 — Thu, 11 Jun 2026 07:04:18 +0000

Agree. Appreciate the reminder.

Ask HN: Want to build something open source on nights and weekends together?

vira28 — Thu, 11 Jun 2026 06:31:00 +0000

Not a hiring post. I have a 9-5 but also a lot of ideas I want to build. The core is just to show up and make small progress — but I find I'm missing accountability and partnership, so I thought it'd be nice to partner with someone. Mostly backend/infra stuff. If you're interested, email in my profile.

Comments URL: https://news.ycombinator.com/item?id=48486932

Points: 39

# Comments: 18

New comment by vira28 in "PgDog is funded and coming to a database near you"

vira28 — Thu, 11 Jun 2026 06:20:17 +0000

Tomas Vondra, a major Postgres contributor recently revived a thread on using Bloom filters - https://www.postgresql.org/message-id/flat/5cd8c20c-14b5-4b0...

So there is more core work happening on support OLAP but I do think it will take some time.

In the meantime, I think we have all the pieces (storage, query engine, table format) to set up a true OLAP. For instance, I created https://github.com/viggy28/streambed to pressure test this idea.

New comment by vira28 in "Show HN: Streambed – Stream Postgres to Iceberg on S3, Supports Postgres Wire"

vira28 — Thu, 04 Jun 2026 05:32:50 +0000

The idea is pretty similar. As per their README, Iceberg support is deprecated.

New comment by vira28 in "Show HN: Streambed – Stream Postgres to Iceberg on S3, Supports Postgres Wire"

vira28 — Thu, 04 Jun 2026 05:30:03 +0000

From what I understand Ducklake needs a dedicated metadata database and it also ties to DuckDB land wherease with Iceberg many engines can query directly.

New comment by vira28 in "Show HN: Streambed – Stream Postgres to Iceberg on S3, Supports Postgres Wire"

vira28 — Wed, 03 Jun 2026 02:47:06 +0000

The challenge with any CDC is making it reliable. Curious, how are you exporting to S3? - Debezium or some service in AWS or home grown tool?

New comment by vira28 in "Show HN: Streambed – Stream Postgres to Iceberg on S3, Supports Postgres Wire"

vira28 — Tue, 02 Jun 2026 05:47:32 +0000

Currently, Strembed expects REPLICA IDENTITY FULL for getting the before and after value of TOAST column. Since we have the data in object storage, we could populate it without the need for REPLICA IDENTITY FULL. Created an issue https://github.com/viggy28/streambed/issues/25 to track this feature.

New comment by vira28 in "Show HN: Streambed – Stream Postgres to Iceberg on S3, Supports Postgres Wire"

vira28 — Tue, 02 Jun 2026 04:51:14 +0000

Welcome. To avoid primary running out of disk space, you can configure max_slot_wal_keep_size https://www.postgresql.org/docs/17/runtime-config-replicatio...

Since Supabase is vanilla Postgres, streambed should work with replica as the source.

reg, Planetscale, I haven't looked at their offerings yet.

Where do you host your DB currently? Happy to try out with that provider as the source.

New comment by vira28 in "Show HN: Streambed – Stream Postgres to Iceberg on S3, Supports Postgres Wire"

vira28 — Mon, 01 Jun 2026 07:27:21 +0000

Aside from the cost, my major motivation is to keep the infrastructure simple. The data is already there in Postgres, so I didn't want to add another data warehouse. I have also shared my thoughts on where this is heading https://viggy28.dev/article/postgres-gateway-drug/

New comment by vira28 in "Show HN: Streambed – Stream Postgres to Iceberg on S3, Supports Postgres Wire"

vira28 — Mon, 01 Jun 2026 07:22:03 +0000

Thanks for the kind words!

Short answer: yes, column-level schema changes sync to Iceberg automatically[0].

Logical replication (pgoutput in v1) doesn't actually stream DDL statements. Instead, Postgres emits a fresh Relation message describing the table's current column layout right before the next change to that table. So we diff that against the last layout we knew and infer what changed.

From there we evolve the Iceberg schema in place: flush any buffered rows under the old schema first, then write a new metadata version with the change. What's handled today:

  - ADD COLUMN — new field ID allocated; the column's Postgres DEFAULT is carried into Iceberg's initial-default/write-default, so existing rows read back correctly
  - DROP COLUMN — removed from the current schema, existing data files untouched
  - Type widening — int4→int8, float4→float8 (the changes Iceberg considers compatible)
  - REPLICA IDENTITY changes

[0] https://github.com/viggy28/streambed/pull/21

New comment by vira28 in "Show HN: Streambed – Stream Postgres to Iceberg on S3, Supports Postgres Wire"

vira28 — Mon, 01 Jun 2026 07:08:01 +0000

Welcome. Would love to hear your experience. Feel free to share here or in the repo. Fully open source.

New comment by vira28 in "Show HN: Streambed – Stream Postgres to Iceberg on S3, Supports Postgres Wire"

vira28 — Mon, 01 Jun 2026 07:07:08 +0000

Should be fairly doable using binlog-based producer https://github.com/go-mysql-org/go-mysql.

New comment by vira28 in "Show HN: Streambed – Stream Postgres to Iceberg on S3, Supports Postgres Wire"

vira28 — Mon, 01 Jun 2026 02:40:40 +0000

Both projects are relevant. Curious, what kinda pushdown capabilities that you were looking for?

New comment by vira28 in "Show HN: Streambed – Stream Postgres to Iceberg on S3, Supports Postgres Wire"

vira28 — Sun, 31 May 2026 23:51:33 +0000

Hello, I checked ingestr repo, and it is in my bookmark. Small world.

Agree, CDC is like Death by a thousand cuts. I believe Debezium has a Java library.

My initial need was Postgres compatibilty. Wanted to give an endpoint that BI and dashboard teams can use to query as if they are querying a Postgres replica. Added more context here https://news.ycombinator.com/item?id=48350820

New comment by vira28 in "Show HN: Streambed – Stream Postgres to Iceberg on S3, Supports Postgres Wire"

vira28 — Sun, 31 May 2026 23:23:51 +0000

Author here. For context, I was the tech lead for the Postgres team at Cloudflare, and this came directly out of a challenge I kept hitting there: BI and dashboard teams needed to run long-running analytical queries, and the answer was always to spin up another bespoke read replica or stand up an ETL dump into an analytical database and query that.

So the question I started with was: what's the fewest components I could get away with? That led to the architecture here — Streambed connects to Postgres as a logical replication subscriber (same mechanism as a read replica) and streams WAL changes straight into Apache Iceberg on S3, queryable from psql via an embedded DuckDB. There are a lot of edge cases to handle, and it's very much early days.

Welcome any feedback.

Show HN: Streambed – Stream Postgres to Iceberg on S3, Supports Postgres Wire

vira28 — Sun, 31 May 2026 18:43:05 +0000

Article URL: https://github.com/viggy28/streambed

Comments URL: https://news.ycombinator.com/item?id=48348429

Points: 129

# Comments: 40

New comment by vira28 in "GitLab announces workforce reduction and end of their CREDIT values"

vira28 — Tue, 12 May 2026 18:40:52 +0000

If I have to guess on why they are providing voluntary exit option, because they have a lot more folks from Europe.

Show HN: pg_savior: a seatbelt for Postgres – blocks accidental DELETE/UPDATE

vira28 — Sun, 26 Apr 2026 17:23:44 +0000

Article URL: https://github.com/viggy28/pg_savior

Comments URL: https://news.ycombinator.com/item?id=47912022

Points: 3

# Comments: 0