Hacker News: danlark1

New comment by danlark1 in "Show HN: misa77 - a codec that decodes 2x faster than LZ4 (at better ratios)"

danlark1 — Wed, 15 Jul 2026 18:52:39 +0000

It's a somewhat known tradeoff, you can streamline and make the format friendlier to do memcpy which this library targets, the more memcpys you do, the faster it is overall to decode. On highly compressible data lz4, snappy become faster. Snappy on level 2 has faster decompression speed

But you have to pay the price that you need a slower encoding, because finding matches, putting restrictions on match lengths, putting things in different streams have costs you need to pay upfront.

Anyway good work, there is probably a need for that.

// Currently own Google's snappy and do compression at Google

P.S. if you want better snappy's results, compile with clang.

P.S.S you can optimize aarch64 speed by movemasks from shrn instruction. https://developer.arm.com/community/arm-community-blogs/b/se...

New comment by danlark1 in "Performance Hints"

danlark1 — Sat, 20 Dec 2025 06:34:49 +0000

Surprisingly I didn't put 2023, it was merged with another submission possibly with the help of mods

Performance Hints

danlark1 — Fri, 19 Dec 2025 17:14:42 +0000

Article URL: https://abseil.io/fast/hints.html

Comments URL: https://news.ycombinator.com/item?id=46328274

Points: 143

# Comments: 43

New comment by danlark1 in "An overengineered solution to `sort | uniq -c` with 25x throughput (hist)"

danlark1 — Mon, 27 Oct 2025 14:34:16 +0000

Yes, sorry, it should be "disclosure"

New comment by danlark1 in "An overengineered solution to `sort | uniq -c` with 25x throughput (hist)"

danlark1 — Mon, 27 Oct 2025 11:18:41 +0000

Disclaimer: the author of the comment is the founder and CTO of ClickHouse

New comment by danlark1 in "86 GB/s bitpacking with ARM SIMD (single thread)"

danlark1 — Sun, 05 Oct 2025 13:58:53 +0000

Great work!

Popular narrative that NEON does not have a move mask alternative. Some time ago I published an article to simulate popular bit packing use cases with NEON with 1-2 instructions. This does not include unpacking cases but can be great for real world applications like compare+find, compare+iterate, compare+test.

https://community.arm.com/arm-community-blogs/b/servers-and-...

New comment by danlark1 in "ClickHouse gets lazier and faster: Introducing lazy materialization"

danlark1 — Tue, 22 Apr 2025 19:10:06 +0000

I am the author of the optimization of partial sorting and selection in Clickhouse. It uses Floyd-Rivest algorithm and we tried a lot of different things back at the time, read [1]

Overall clickhouse reads blocks of fixed sizes (64k) and finds top elements and then does top of the top until it converges.

[1] https://danlark.org/2020/11/11/miniselect-practical-and-gene...

New comment by danlark1 in "Optimizing 128-bit Division"

danlark1 — Sun, 19 Jul 2020 22:21:42 +0000

That's a difficult question. As I am working at Google I need to consult each open source I want to publish on my behalf. This does not involve contributing to the list of the approved projects and telling about these contributions.

If you want to workaround this for now, I suggest looking into libdivide (https://github.com/ridiculousfish/libdivide), it is published with the boost license and the library contains all the needed artifacts I described in the article (unfortunately, not combined).

New comment by danlark1 in "Optimizing 128-bit Division"

danlark1 — Sun, 19 Jul 2020 20:40:46 +0000

For compiler-rt it is Apache 2.0 license

https://github.com/llvm/llvm-project/blob/master/compiler-rt...

New comment by danlark1 in "Optimizing 128-bit Division"

danlark1 — Sun, 19 Jul 2020 11:54:44 +0000

Hi, I thought that this is not the most interesting part of all the benchmarks, for example, all we need to test that with the quotient and the remainder: dividend = quotient * divisor + remainder, remainder < divisor and multiplication does not overflow which is free of division operations.

Yet, I added several tests like dividend < divisor, close to zero remainders, a lot of random stuff just to make sure each time I add a new approach, it is correct.

New comment by danlark1 in "Optimizing 128-bit Division"

danlark1 — Sun, 19 Jul 2020 11:03:47 +0000

Hi, I believe they can be done for 32 bit platforms also with the multiword division in Knuth https://skanthak.homepage.t-online.de/division.html what I've chosen for fallback if not x86_64 platform.

I will try to implement the same optimizations in Rust in the upcoming weeks

UPD And we opened an issue :) https://github.com/rust-lang/compiler-builtins/issues/368

New comment by danlark1 in "Optimizing 128-bit Division"

danlark1 — Sun, 19 Jul 2020 10:16:55 +0000

Hi everyone, the author is here. Yes, I believe the title should be changed to `Optimizing 128-bit Division`

Yet, I was not expecting it to be here. Overall, I put some knowledge hidden in Hacker's Delight book, Knuth, GMP and GNU in the article with my knowledge of low level optimizations. In the end it turned out to be a good thing to write and to submit into LLVM