From the post:

> As an example, the values of three out of 20 columns of the NYC taxi ride data set (a subset of 119 files overall, ~9.2 GB total, ~650M rows) can be summed up in ~2.7 sec using the row reader API with indexed access on my MacBook Pro M3 Max with 16 CPU cores. With the column reader API, the same task takes ~1.2 sec.

In my measurements, this is significantly faster than parquet-java for the same task (which is not surprising, as Hardwood is multi-threaded); but I want to be sure I am setting up and configuring parquet-java correctly before publishing any comparisons. The test above also is hooked up to run parquet-java (and there's a set-up for PyArrow, too), so you could run it yourself on your machine if you wanted to.

So far, we've spent most time optimizing for flat (non-nested) data sets which are fully parsed (either all columns, or with projections) and I think it's faring really well for those. There's no support for predicate push-down yet, so right now, Hardwood isn't optimal for use cases with high query selectivity; this is the next thing on the roadmap though.

Approaching the home stretch for a first 1.0 preview release, including: support for parsing Parquet files with flat and nested schemas, all physical and logical column types, core and advanced encodings, projections, compression, multi-threading, etc. all that with a pretty decent performance.

Next on the roadmap are SIMD support, predicate push-down (bloom filters, statistics, etc.), writer support.

Reading through the README, this piqued my curiosity:

> Small or fast transactions may share the same WAL position.

I don't think that's true; each data change and each commit (whether explicit or not) has its own dedicated LSN.

> LSNs should be treated as monotonic but not dense.

That's not correct; commit LSNs are monotonically increasing, and within a transaction, event LSNs are monotonically increasing. I.e. the tuple commit-LSN/event-LSN is monotonically increasing, but not LSNs per se. You can run multiple concurrent transactions to observe this.

Made it to #369 in 2025 with morling.dev; let's see what's in stock this year :)

  year  total_score  rank  days_mentioned
  2025  903          369   8
  2024  604          581   2
  2023  547          861   3
  2022  450          1165  4
  2021  188          2308  2

But also when running on the JVM, things have improved dramatically over the last few years, e.g. due to things such as AOT class loading and linking. For instance, a single node Kafka broker starts in ~300 ms.

> Pkl, which is at least built using Graal Native Image, but (IMO) would _still_ have better adoption if it was written in something else.

Why do you think is this?

Installation is via brew, so same experience as for all the other CLI tools you're using. The binary size is on the higher end (52 MB), but I don't think this makes any relevant difference for practical purposes. Build times with GraalVM are still not ideal (though getting better). Cross compilation is another sore point, I'm managing it via platform-specific GitHub Action runners. From a user perspective, non of this matters, I'd bet most users don't know that kcctl is written in Java.

That's not to say I don't use LLMs at all; I don't use them for original writing though, but rather as a copy editor, helping with improving expressions, getting language and grammar right, etc.

I have publicly documented my usage of AI for writing on my blog: https://www.morling.dev/ai/. I wished more folks would do that, as I think transparency is key here, every reader should always know whether they'rea reading a text written by a human or by a machine.

Comments URL: https://news.ycombinator.com/item?id=46305311

Points: 4

# Comments: 1

Comments URL: https://news.ycombinator.com/item?id=46266627

Points: 5

# Comments: 1

All this is not great for the audience (who have "invested" into your session, by paying for the ticket, spending time away from work and family, not attending other concurrent sessions, etc.), and it can so easily be avoided by rehearsing.

The most common reason I have seen for folks skipping to rehearse is the awkward feeling you might have when speaking loud all by yourself. If that's the issue, it can help to do a dry run in front of colleagues. In any case, "winging it" is best reserved for later on, after having gathered quite a bit of speaking experience and having spoken about the same, or very similar, topics before.

I'd also recommend to avoid reading from slides during a talk as much as possible, it's also not a great experience for the audience. There shouldn't be much text on slides to begin with, as folks will either read that, or listen to what you say, but typically have a hard time doing both at once.

(All this is a general recommendation, not a comment on your talks which I have not seen)

> This is why we have indexes in databases, which, if you squint a little, are just another kind of materialized view, at least in their covering form.

[1] https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/...

I am discussing this approach, just not under that name:

> Gaps in the sequence are fine, hence it is possible to increment the persistent state of the sequence or counter in larger steps, and dispense the actual values from an in-memory copy.

In that model, a database sequence (e.g. fetched in 100 increments) represents the hi value, and local increments to the fetched sequence value are the low value.

However, unlike the log-based approach, this does not ensure monotonicity across multiple concurrent requests.