Hacker News: pmm

New comment by pmm in "My Browser WASM't Prepared for This. Using DuckDB, Apache Arrow and Web Workers"

pmm — Mon, 07 Apr 2025 09:57:50 +0000

Author here. Thank you all for the comments. I take full responsibility for stupidly using an image for posting the code snippet. Sorry for that! Also, the article was originally posted almost 2 years ago (and "resurrected" with the recent migration to Medium). This is why a fairly old DuckDB version is referenced there. Some of the issues I observed are now gone too.

Obviously, many things have changed since then. We've experimented extensively and moved back and forth with using DuckDB for our internal cloud processing architecture. We eventually settled on just using it for reading the data and then handling everything else in custom workers. Even using TypeScript, we achieved close to 1M events/s per worker overall with very high scalability. However, our use-case is quite distinct. We use a custom query engine (for sequence processing), which has driven many design decisions.

Overall, I think DuckDB (both vanilla and WASM version) is absolutely phenomenal. It also matured since my original blog post. I believe we'll only see more and more projects using it as their backbone. For example, MotherDuck is doing some amazing things with it (e.g., https://duckdb.org/2023/03/12/duckdb-ui) but there are also many more exciting initiatives.

Making LLMs Ruthless – AI Alignment Is More Fragile Than You Think

pmm — Fri, 07 Feb 2025 12:35:40 +0000

Article URL: https://substack.com/home/post/p-156320854

Comments URL: https://news.ycombinator.com/item?id=42971863

Points: 1

# Comments: 1

How to build a query engine with Automata, TypeScript and Apache Arrow

pmm — Tue, 22 Oct 2024 16:26:39 +0000

Article URL: https://www.motifanalytics.com/posts/how-to-build-a-query-engine

Comments URL: https://news.ycombinator.com/item?id=41915942

Points: 2

# Comments: 0

My browser WASM't prepared for this. DuckDB, Arrow and Web Workers in real life

pmm — Wed, 29 Nov 2023 18:24:36 +0000

Article URL: https://www.motifanalytics.com/blog/my-browser-wasmt-prepared-for-this-on-challenges-of-using-duckdb-apache-arrow-and-web-workers-together

Comments URL: https://news.ycombinator.com/item?id=38463077

Points: 9

# Comments: 1

New comment by pmm in "Every database will become a vector database sooner or later"

pmm — Tue, 03 Oct 2023 06:24:34 +0000

I think just as with full-text search, vector search, if supported, will be full of tradeoffs for general purposes databases.

The view that everything needs to support direct input for generative AI is short sighted. There are other use cases as well. Even if ultimately these will become just building blocks for whatever AGI there comes. Horses for courses

New comment by pmm in "Fast and flexible observability with canonical log lines"

pmm — Tue, 30 Jul 2019 22:37:23 +0000

I feel that logs are around for so long that it's easy to take their capabilities for granted and not go much further. This is another example there's more that can be done. It reminds me rfc5424

At LogSense.com we actually tackled this problem too and came with automatic pattern discovery that pretty much converts all logs into structured data. I actually just posted it here: https://news.ycombinator.com/item?id=20569879 I am really curious if this is something that you consider helpful and any feedback is very welcome

Why Pattern Discovery Matters

pmm — Tue, 30 Jul 2019 22:31:40 +0000

Article URL: https://www.logsense.com/blog/why-pattern-discovery-matters

Comments URL: https://news.ycombinator.com/item?id=20569879

Points: 21

# Comments: 1

New comment by pmm in "Similarities Between the Lion Air and Ethiopian 737 Max Crashes"

pmm — Sun, 10 Mar 2019 21:57:03 +0000

Right now the MAX version is at least an order of magnitude less safe than the previous generation (put some rough estimated in another comment)

New comment by pmm in "Similarities Between the Lion Air and Ethiopian 737 Max Crashes"

pmm — Sun, 10 Mar 2019 21:12:54 +0000

There were 41k flights until May 2018 [1]. I was not able to find the current number but with 350 models delivered so far and 4 average flights per day that would sum up to ~500k flights in total at most (but more probably half of that due to ramping up the deliveries).

With 2 accidents the rate is now 4/million. That's at least an order of magnitude difference more than the average of the modern jetliners [2]

Of course, with two events it might be just bad luck but the similarities are concerning, as pretty much everyone observes here

[1] https://randy.newairplane.com/2018/05/22/737-max-a-year-of-s... [2] http://www.airsafe.com/events/models/rate_mod.htm

New comment by pmm in "Go vs. Rust? Choose Go"

pmm — Sun, 17 Sep 2017 11:53:15 +0000

> Predictable runtime behavior (zero cost abstractions and no garbage collector).

I see a lot of people considering Rust/Go because they are looking for something much more modern than C++ and at the same time not running on JVM due to the above