Just download the single binary, for most platforms, and run "weed mini -dir=your_data_directory", with all the configuration optimized.

I am trying to support AWS S3 APIs as complete as possible.

Recently added support for Table Bucket, besides myriads of details, such as policies, STS, IAM, OIDC, WORM, lock and versioning, governance, etc.

Haha, you used Claude to find the Clause code.

I used Claude to generate a lot of admin UI pages, saved a lot of time. The core storage engine part I dare not using AI, same as you.

SeaweedFS was started as a learning project and evolves along the way, getting ideas from papers for Facebook Haystack, Google Colossus, Facebook Tectonics. With its distributed append-only storage, it naturally fits object store. Sorry to see MinIO went away. SeaweedFS learned a lot from it. Some S3 interface code was copied from MinIO when it was still Apache 2.0 License. AWS S3 APIs are fairly complicated. I am trying to replicate as much as possible.

Some recent developments:

* Run "weed mini -dir=xxx", it will just work. Nothing else to setup.

* Added Table Bucket and Iceberg Catalog.

* Added admin UI

Why skipping SeaweedFS? It rank #1 on all benchmarks, and has a lot of features.

Just run "weed sever -s3 -dir=..." to have an object store.

Recently SeaweedFS is moving fast and added a lot more features, such as: * Server Side Encryption: SSE-S3, SSE-KMS, SSE-C * Object Versioning * Object Lock & Retention * IAM integration * a lot of integration tests

Also, SeaweedFS performance is the best in almost all categories in a user's test https://www.repoflow.io/blog/benchmarking-self-hosted-s3-com... And after that, there is a recent architectural change that increases performance even more, with write latency reduced by 30%.

Sorry, everybody has different background of knowledge. Hard to understand where the question comes from. I think https://www.usenix.org/system/files/fast21-pan.pdf may be helpful here.

A blob has its own storage, which can be replicated to other hosts in case current host is not available. It can scale up independently of the file metadata.

Files and S3 are higher layers above the blob storage. They require metadata to manage to the blobs, and other metadata for directories, S3 access, etc.

These metadata usually sit together with the disks containing the files. But in highly scalable systems, the metadata has dedicated stores, e.g., Google's Colossus, Facebook's Techtonics, etc. SeaweedFS file system layer is built as a web application of managing the metadata of blobs.

Actually SeaweedFS file system implementation is just one way to manage the metadata. There are other possible variations, depending on requirements.

There are a couple of slides on the SeaweedFS github README page. You may get more details there.

The allocated storage is append only. For updates, just allocate another blob. The deleted blobs would be garbage collected later. So it is not really mmap.

> Also what is the difference between a file, an object, a blob, a filesystem and an object store?

The answer would be too long to fit here. Maybe chatgpt can help. :)

> Is all this just files indexed with sql?

Sort of yes.

One similar use case used Cassandra as SeaweedFS filer store, and created thousands of files per second in a temp folder, and moved the files to a final folder. It caused a lot of tombstones for the updates in Cassandra.

Later, they changed to use Redis for the temp folder, and keep Cassandra for other folders. Everything has been very smooth since then.