Hacker News: richiejp

New comment by richiejp in "Why we write our own C and C++ inference engines"

richiejp — Mon, 03 Aug 2026 19:42:36 +0000

Getting any performance improvement over vLLM is just a bonus although the RAM reductions are very nice. The main point is that we find deploying Python based projects to be a pain. I can't imagine how you know what files are unreachable in a dynamic language, but in any case that doesn't begin to solve issues like PyTorch requiring a C++ compiler at runtime to generate optimised CPU code. This isn't exactly embedded device friendly.

Having said that if you know of some magic bullet to fix the problems with the Python ecosystem (that vLLM and Pytorch are apparently unaware of) then I will be happy to try it because our CI is constantly blocked by Python related issues.

New comment by richiejp in "Why we write our own C and C++ inference engines"

richiejp — Mon, 03 Aug 2026 18:48:34 +0000

I can only assume you are talking about LocalAI (not vllm.cpp) which is modular and contains adapters for many different AI engines written in various languages. LocalAI's core is written in pure Go and you choose what engines to install at runtime.

vLLM.cpp's runtime is pure C++, but AOT Triton compilation is done at build time in Python.

C++ Vs Rust: Which is better for writing AI/ML code with LLMs

richiejp — Sun, 26 Jul 2026 21:50:30 +0000

Article URL: https://richiejp.com/rust-vs-cpp-llm-coded-model-conversion-from-pytorch

Comments URL: https://news.ycombinator.com/item?id=49062785

Points: 2

# Comments: 0

Show HN: LocalVQE: Tiny ~1M param audio model that cancels echo and noise

richiejp — Fri, 01 May 2026 12:21:31 +0000

Article URL: https://huggingface.co/spaces/LocalAI-io/LocalVQE-demo

Comments URL: https://news.ycombinator.com/item?id=47973955

Points: 1

# Comments: 0

New comment by richiejp in "[dead]"

richiejp — Fri, 02 May 2025 08:44:29 +0000

https://github.com/mudler/LocalAGI

Show HN: WebGPU Bitbanging 1D Reversible Automata

richiejp — Mon, 06 Jan 2025 17:55:23 +0000

Recently I posted on here my automata implementation: https://news.ycombinator.com/item?id=42401976

It seems like hackers liked it and I got some interesting comments, so I decided to finally convert it to something that can run in the browser.

After this I'm interested in higher dimensional automata, but I vaguely remember from reading Wolfram's book that adding more dimensions makes it harder to find interesting rules, although it is possible to scan generated results and measure the entropy to see if it looks interesting. But anyway does anyone have advice or pointers on 3D automata?

Comments URL: https://news.ycombinator.com/item?id=42613063

Points: 1

# Comments: 0

New comment by richiejp in "Bitbanging 1D Reversible Automata"

richiejp — Fri, 13 Dec 2024 06:51:08 +0000

Yes it does. Even discarding arguments that different rules in the same class are the same automaton and multiple cells are the same automaton, the article includes reversible and non-reversible automata which are distinct classes.