Hacker News: huntaub

New comment by huntaub in "What are you using for distributed block storage on NVMe in production?"

huntaub — Thu, 04 Jun 2026 18:08:49 +0000

This is the problem that we're building for at Archil [1]. Would be super curious to hear more about your use case (and specifically why you're not able to use the hyperscaler managed offerings). Feel free to email me at hleath [at] archil.com

[1] https://archil.com

Bash is the SQL for file systems

huntaub — Thu, 16 Apr 2026 16:34:16 +0000

Article URL: https://archil.com/post/serverless-execution

Comments URL: https://news.ycombinator.com/item?id=47795906

Points: 5

# Comments: 0

S3 Files: the right product on the wrong foundation

huntaub — Mon, 13 Apr 2026 23:02:09 +0000

Article URL: https://archil.com/post/s3-files-deep-dive

Comments URL: https://news.ycombinator.com/item?id=47759082

Points: 4

# Comments: 0

New comment by huntaub in "Improving storage efficiency in Magic Pocket, Dropbox's immutable blob store"

huntaub — Thu, 09 Apr 2026 15:10:06 +0000

I don't think there's much for Amazon to gain from publishing these sorts of internal details. Amazon's services are used by developers who are looking to tightly optimize their usage. If Amazon were to publish detailed internal information, it's likely that folks would start optimizing applications based on internal details that have the potential to change over time.

Secondly, I think that a lot of companies publish these "tech blogs" as a way to boost recruiting (look at the cool stuff that we're doing, don't you want to join us?). Amazon, of course, doesn't have a recruiting problem. If you want to work on the largest-scale systems, it's already a top destination for you.

New comment by huntaub in "S3 Files"

huntaub — Wed, 08 Apr 2026 10:24:23 +0000

I imagine (hope) that they are doing some kind of intelligent read-ahead in the frontend servers to optimize for sequential reads that would avoid this looking terrible for applications.

New comment by huntaub in "S3 Files"

huntaub — Wed, 08 Apr 2026 10:21:50 +0000

What does DuckDB need that NFS/SMB do not provide?

New comment by huntaub in "S3 Files"

huntaub — Wed, 08 Apr 2026 10:20:30 +0000

Of course, you don't need to, this is just a way to opt-in to getting file semantics on top of S3.

The purpose of S3 isn't to be cheap, it's to be simple.

New comment by huntaub in "S3 Files"

huntaub — Wed, 08 Apr 2026 10:19:34 +0000

Notably, this is going to manage your data in it's native format (i.e. you can actually read-write the files out of the S3 bucket as if they were actual objects, mapping 1:1 to each file). The ZFS backend is (almost certainly) a block-based format that is persisted to S3 (meaning that you cannot use it for existing data in S3, and you cannot access data written through ZFS via S3).

New comment by huntaub in "S3 Files"

huntaub — Tue, 07 Apr 2026 20:52:02 +0000

This is pretty different than s3fs. s3fs is a FUSE file system that is backed by S3.

This means that all of the non-atomic operations that you might want to do on S3 (including edits to the middle of files, renames, etc) are run on the machine running S3fs. As a result, if your machine crashes, it's not clear what's going to show up in your S3 bucket or if would corrupt things.

As a result, S3fs is also slow because it means that the next stop after your machine is S3, which isn't suitable for many file-based applications.

What AWS has built here is different, using EFS as the middle layer means that there's a safe, durable place for your file system operations to go while they're being assembled in object operations. It also means that the performance should be much better than s3fs (it's talking to ssds where data is 1ms away instead of hdds where data is 30ms away).

New comment by huntaub in "Launch HN: Terminal Use (YC W26) – Vercel for filesystem-based agents"

huntaub — Mon, 09 Mar 2026 17:56:20 +0000

howdy! two things on the archil front:

1. we're not NFS, we wrote our own protocol to get much better performance

2. we're planning on coming out with native branching this month, which should make these kinds of workloads much easier to build!

New comment by huntaub in "Supertoast tables"

huntaub — Fri, 06 Mar 2026 22:20:17 +0000

Well, I think this is what our company, Archil, is working on. We basically built an SSD clustering layer that proxies/caches/and assembles requests into object storage so that you can run a POSIX file system directly on top.

There's also some really great projects like SlateDB in this space, which could be more like what you're looking for (~RocksDB like API that runs on S3).

New comment by huntaub in "Just-bash: Bash for Agents"

huntaub — Thu, 26 Feb 2026 15:18:23 +0000

We just released a driver that allows users of just-bash to attach a full Archil file system, synced to S3. This would let you run just-bash in an enrivonment where you don't have a full VM and get high-performance access to data that's in your S3 bucket already to do like greps or edits.

Check it out here: https://www.npmjs.com/package/@archildata/just-bash

New comment by huntaub in "GitHub is down again"

huntaub — Mon, 09 Feb 2026 17:08:43 +0000

It's 100% because the number of operations happening on Github has likely 100x'd since the introduction of coding agents. They built Github for one kind of scale, and the problem is that they've all of a sudden found themselves with a new kind of scale.

That doesn't normally happen to platforms of this size.

New comment by huntaub in "Don't rent the cloud, own instead"

huntaub — Thu, 05 Feb 2026 13:03:27 +0000

This turns out to be a more and more important primitive for companies who are building their own models [1].

[1] https://si.inc/posts/the-heap/

Connect just-bash directly to data in S3 with Archil

huntaub — Mon, 02 Feb 2026 20:19:22 +0000

Article URL: https://archil.com/post/just-bash-support

Comments URL: https://news.ycombinator.com/item?id=46860868

Points: 1

# Comments: 0

Why file systems are here to stay for agents

huntaub — Mon, 19 Jan 2026 22:37:27 +0000

Article URL: https://archil.com/post/why-file-systems-are-here-to-stay

Comments URL: https://news.ycombinator.com/item?id=46685481

Points: 1

# Comments: 0

New comment by huntaub in "Archil: An elastic, scale-out file system that syncs to S3"

huntaub — Fri, 16 Jan 2026 15:21:43 +0000

It does! We see a lot of users choose Cloudflare's R2 because it has no egress fees and then pair it with Archil so they get fast, local caching of their content from wherever their compute lives.

Archil: An elastic, scale-out file system that syncs to S3

huntaub — Fri, 16 Jan 2026 15:09:33 +0000

Article URL: https://archil.com

Comments URL: https://news.ycombinator.com/item?id=46647119

Points: 4

# Comments: 3

New comment by huntaub in "JuiceFS is a distributed POSIX file system built on top of Redis and S3"

huntaub — Thu, 15 Jan 2026 22:43:03 +0000

Our product is Archil [1], and we are building our service on top of a durable, distributed SSD storage layer. As a result, we have the ability to: (a) store and use data in S3 in its native format [not a block based format like the other solutions in this thread], (b) durably commit writes to our storage layer with lower latency than products which operate as installable OSS libraries and communicate with S3 directly, and (c) handle multiple writers from different instances like NFS.

Our team spent years working on NFS+Lustre products at Amazon (EFS and FSx for Lustre), so we understand the performance problems that these storage products have traditionally had.

We've built a custom protocol that allows our users to achieve high-performance for small file operations (git -- perfect for coding agents) and highly-parallel HPC workloads (model training, inference).

Obviously, there are tons of storage products because everyone makes different tradeoffs around durability, file size optimizations, etc. We're excited to have an approach that we think can flex around these properties dynamically, while providing best-in-class performance when compared to "true" storage systems like VAST, Weka, and Pure.

[1] https://archil.com

New comment by huntaub in "JuiceFS is a distributed POSIX file system built on top of Redis and S3"

huntaub — Thu, 15 Jan 2026 22:22:16 +0000

tl;dr it doesn't. I'm not sure what they're planning in this capacity (I haven't checked out sprites myself), but I would guess that it's going to be a function of "snapshots" as a mechanism to give multiple clients ephemeral write access to the same disk.