Awesome to hear that you're excited about Zig.

Thanks for sharing the link to our repo also—would love to take you on a 1-on-1 tour of the codebase sometime if you'd be up for that!

The first was fixed within 24 hours, in fact just before I reported it. The others had clear TODO error messages in the panic, and there were easy enough workarounds.

Comments URL: https://news.ycombinator.com/item?id=33328147

Points: 75

# Comments: 114

It's also made TigerBeetle's code more reliable, because tests can assert that limits are never exceeded. This has detected rare leaks that might otherwise have only been detected in production.

The limit of 70 lines is actually a slight increase beyond the 60 line limit imposed by NASA's Power of Ten Rules for Safety Critical Software.

In my experience, in every instance where we've refactored an overlong function, the result has almost always been safer.

We use what we have available, according to the context: checksums, assertions, hash chains. You can't always use every technique. But anything that can possibly be verified online, we do.

Buffer bleeds also terrify me. In fact, I worked on static analysis tooling to detect zero day buffer bleed exploits in the Zip file format [1].

However, to be clear, the heart of a bleed is a logic error, and therefore even memory safe languages such as JavaScript can be vulnerable.

[1] https://news.ycombinator.com/item?id=31852389

Here's an overview with references to the simulator source, and links to resources from FoundationDB and Dropbox: https://github.com/tigerbeetledb/tigerbeetle/blob/main/docs/...

We also have a $20k bounty that you can take part in, where you can run the simulator yourself.

Joran from the TigerBeetle team here.

Appreciate your balanced comment.

To be fair, we're certainly concerned about logic errors and buffer bleeds. The philosophy in TigerBeetle is always to downgrade a worse bug to a lesser. For example, if it's a choice between correctness and liveness, we'll downgrade the potential correctness bug to a crash.

In the specific case of message buffer reuse here, our last line of defense then is also TigerBeetle's assertions, hash chains and checksums. These exhaustively check all function pre/post-conditions, arguments, processing steps and return values. The assertion-function ratio is then also tracked for coverage, especially in critical sections like our consensus or storage engine.

So—apologies for the wince! I feel it too, this would certainly be a nasty bug if it were to happen.

We have a secret plan for this too. ;)

Static allocation does make for some extremely hard guarantees on p100 latency. For example, for a batch of 8191 queries, the performance plot is like Y=10ms, i.e. a flat line.

And memory doesn't increase as throughput increases—it's just another flat line.

I personally find it also to be a fun way of coding, everything is explicit and limits are well-defined.

Data Oriented Design runs like a river through TigerBeetle—it's always on our mind.

By the way, have you seen Andrew Kelley's Handmade Seattle talk on Practical DOD? [1]

[1] https://vimeo.com/649009599

Joran from the TigerBeetle team here.

> I would imagine this would be potentially more efficient than having conventional mallocs.

Yes, in our experience, static allocation means we use sometimes 10x less memory. For example, TigerBeetle's storage engine can theoretically address up to 100 TiB with less than 1 GiB of RAM for the LSM in-memory manifest, which is efficient.

Because we think about memory so much upfront, in the design phase, we tend not to waste it.

Joran from the TigerBeetle team here.

TigerBeetle's storage engine is designed also for range queries, and we have some interesting ideas for our query engine in the works.

To be sure, there are some tricky things that crop up, such as pipeline blockers for queries. However, we do have limits on literally everything, so that makes it easier—static allocation is best when it's done viral.

We wanted to go into so much more detail on all this but there was only space for so much. Stay tuned.

TigerBeetle uses Deterministic Simulation Testing to test and keep testing these paths. Fuzzing and static allocation are force multipliers when applied together, because you can now flush out leaks and deadlocks in testing, rather than letting these spillover into production.

Without static allocation, it's a little harder to find leaks in testing, because the limits that would define a leak are not explicit.

Joran from the TigerBeetle team here.

This was in fact one of our motivations for static allocation—thinking about how best to handle overload from the network, while remaining stable. The Google SRE book has a great chapter on this called "Handling Overload" and this had an impact on us. We were thinking, well, how do we get this right for a database?

We also wanted to make explicit what is often implicit, so that the operator has a clear sense of how to provision their API layer around TigerBeetle.

As you zoom in, you will see differences. For example, TigerBeetle's data structures are all cache line aligned, and we use static allocation etc. The storage fault model is significantly different.

We use the same testing techniques though. FDB are pioneers in the space.

Have you seen this post [1] about struct packing? This is what we did for TB's structures, so that we can switch them to `extern struct` for C ABI compatibility. It also helped side step the packed struct bugs.

[1] http://www.catb.org/esr/structure-packing/