Hacker News: benbjohnson

New comment by benbjohnson in "Litestream Writable VFS"

benbjohnson — Wed, 04 Feb 2026 23:43:56 +0000

If you can have multiple writers to a single database then you'd need to look at something like cr-sqlite[1] that uses CRDT to figure out conflicts. If you're just replicating separate databases then you might be able to replicate each one using Litestream and then use SQLite's ATTACH[2] to connect them together. There is a limit on how many databases you can attach together in a session though.

[1]: https://github.com/vlcn-io/cr-sqlite

[2]: https://sqlite.org/lang_attach.html

New comment by benbjohnson in "Litestream Writable VFS"

benbjohnson — Wed, 04 Feb 2026 23:41:31 +0000

Litestream author here. You can use the built-in file replication. It'll replicate all your database changes to another path on disk. I use it a lot for testing things out:

https://litestream.io/guides/file/

New comment by benbjohnson in "Litestream Writable VFS"

benbjohnson — Wed, 04 Feb 2026 23:40:12 +0000

Litestream author here. Currently we're handling the "single writer" issue outside of Litestream. We have controls in our internal systems that make it work well. But yes, the lease PR is the direction we're looking at going.

I'm not sure you can have readers see something separate than writers. When SQLite promotes a read lock to a write lock under WAL then it checks if any of the data has changed and then fails the transaction if it has.

New comment by benbjohnson in "Litestream VFS"

benbjohnson — Thu, 11 Dec 2025 22:34:35 +0000

Author here. It will cache pages in memory right now but it doesn't do disk-based caching currently. That's a good idea though and probably not too difficult to implement. Feel free to post a feature request as a GitHub issue on the project.

New comment by benbjohnson in "Litestream VFS"

benbjohnson — Thu, 11 Dec 2025 21:15:29 +0000

The VFS is read only but it will continuously poll for new updates so if you have a writer somewhere else using regular Litestream then it will pick up those updates automatically.

New comment by benbjohnson in "Litestream VFS"

benbjohnson — Thu, 11 Dec 2025 20:03:51 +0000

Author here. You can query the state of a table from an hour ago with Litestream VFS. It won't give you versioned tables in the sense that every time you update a row that it writes a new version in a table somewhere though.

New comment by benbjohnson in "Litestream VFS"

benbjohnson — Thu, 11 Dec 2025 19:59:36 +0000

Author here. We've done some proof-of-concept work on creating distributed leases using S3. We have some use cases internally where we've considered adding write capabilities to the VFS but we haven't started any work on it yet.

New comment by benbjohnson in "Litestream VFS"

benbjohnson — Thu, 11 Dec 2025 19:57:38 +0000

Author here. Litestream VFS will automatically poll for new back up data every second so it keeps itself up to date with any changes made by the original database.

You don't need any additional code (Python or otherwise) to use the VFS. It will work on the SQLite CLI as is.

New comment by benbjohnson in "Litestream VFS"

benbjohnson — Thu, 11 Dec 2025 19:55:34 +0000

It worked great! Thanks for your work on it.

New comment by benbjohnson in "Litestream VFS"

benbjohnson — Thu, 11 Dec 2025 18:52:03 +0000

Author here. The VFS support right now is currently read only so it's useful for something more like a shared cache of data.

New comment by benbjohnson in "Litestream VFS"

benbjohnson — Thu, 11 Dec 2025 18:51:01 +0000

Author here. Yes, Litestream v0.5.3 has been released with a new read-only VFS option: https://github.com/benbjohnson/litestream/releases/tag/v0.5....

New comment by benbjohnson in "Hold Off on Litestream 0.5.0"

benbjohnson — Tue, 14 Oct 2025 17:47:52 +0000

Litestream author here. Yes, that's correct. LTX was the biggest hurdle to get over and was impossible to switch over incrementally. The storage layer change brings a lot of benefits and enables a lot of future work that we're really excited about.

Thanks to mtlynch and everyone else who has submitted bug reports. We're squashing issues and working to get everything stable as quickly as possible.

New comment by benbjohnson in "Litestream: Revamped"

benbjohnson — Fri, 30 May 2025 16:17:18 +0000

The changes are still in progress. The blog post was just about future work that we're working on. However, we don't have plans to do failover with Litestream at the moment.

New comment by benbjohnson in "Litestream: Revamped"

benbjohnson — Tue, 20 May 2025 22:08:02 +0000

Litestream saves WAL segments to a given time granularity. By default, it ships off WAL changes every second so you should be able to restore to any given second in your history (within your retention period).

New comment by benbjohnson in "Litestream: Revamped"

benbjohnson — Tue, 20 May 2025 22:05:58 +0000

Backups & read replicas are the primary use cases. If you're interested in local-first, you can check out projects like cr-sqlite[1].

[1]: https://github.com/vlcn-io/cr-sqlite

New comment by benbjohnson in "Litestream: Revamped"

benbjohnson — Tue, 20 May 2025 22:04:50 +0000

We don't support plug-ins at the moment but there's several backends at the moment (S3, Azure Blob Storage, Google Cloud Storage, SFTP, etc)

New comment by benbjohnson in "A faster way to copy SQLite databases between computers"

benbjohnson — Thu, 01 May 2025 17:51:14 +0000

Not all corruption is detectable. You could make a copy during a transaction where only a subset of the transactions saved pages are persisted but all branch & leaf pages are pointed to correctly. That would give you a state of the database that never actually existed and break atomicity.

New comment by benbjohnson in "Raft: Understandable Distributed Consensus (2014)"

benbjohnson — Fri, 27 Sep 2024 20:48:28 +0000

I haven't read that paper but it seems like it's fixing a different problem of Byzantine fault tolerance. Most consensus systems that are internal for an organization don't have the Byzantine issue so it simplifies the problem.

New comment by benbjohnson in "Raft: Understandable Distributed Consensus (2014)"

benbjohnson — Fri, 27 Sep 2024 20:46:38 +0000

Thanks so much for letting me know! It's always hard to tell when I put something out there if it just gets lost in the ether. I'm glad to hear it helped so many folks.

New comment by benbjohnson in "Raft: Understandable Distributed Consensus (2014)"

benbjohnson — Fri, 27 Sep 2024 20:45:41 +0000

It's all done with d3 and JavaScript. The visualizations aren't deterministic so I ended up writing a shitty Raft implementation in JS. Overall it was a terrible approach because it was so time consuming but I made it work. You can find all the source code in this repo: https://github.com/benbjohnson/thesecretlivesofdata