No RSS feed currently, but it's a good idea to add one!

Doing this means that you can't just tokenize the string output of the chat template as one big string. You might need to tokenize things separately, and combine them after.

Obviously the prefix-with-backslash convention won't do it. The escaping system could be something like inserting a character on the second position in the text repr, and reversing that on output too if it matches an escaped known special token.

Changing the vocab on the fly requires tokenizing things separately, breaking the chat template.

Anecdotally, even claude code has an anneurism sometimes when listing special tokens. Idk exactly what claude's token is, but I'm fairly sure I've seen it stop generation when it tried to generate it before.

I should also say that I've (clearly) not thought about this deeply. There should be a simpler way to do it.

For this reason, it can be tricky to work on the runtime for a model with the same model. This really feels like an accidental problem, but I'm not sure if it's really solvable without abandoning the text representations altogether (and the jinja abstraction along with it).

Comments URL: https://news.ycombinator.com/item?id=48138332

Points: 161

# Comments: 48

In contrast to Ollama, this is a self-contained library, not a server.

I wrote some quick notes on this blogpost, just to jot down how we think about good open-source citizenship: https://www.nobodywho.ai/posts/notes-on-friends-dont-let-fri...

NobodyWho is making developer tools for running small language models in local-first applications. Our core principle is to ship the model weights along with the application, and then do efficient inference locally and offline, on any device. We run fast on Linux, MacOS, Windows, Android and iOS.

The main product is an inference library that wraps llama.cpp, written in Rust. We provide bindings for Python, Godot (the game engine), and will be releasing a Flutter plugin soon. It's all licensed under EUPL 1.2. Repo here: https://github.com/nobodywho-ooo/nobodywho/

We're hiring people who are comfortable building highly cross-platform FFI applications in Rust (with C++ dependencies), and people who are deeply familiar with language models and the open standards around them, as well as fine-tuning and evaluating models. We're also looking for a technical DevRel profile.

If any of that sounds relevant to you, feel free to email me: a>at ]]>Sat, 03 Jan 2026 14:19:13 +0000https://news.ycombinator.com/item?id=46476963bashbjornhttps://news.ycombinator.com/item?id=46476963https://news.ycombinator.com/item?id=46476963I made something silly to this effect last year:

https://gitlab.com/AsbjornOlling/nixllm

You provide the prompt to a nix function, and it deterministically generates the source code using llama.cpp.

At those sizes, it's great for generating non-repetitive flavortext for NPCs. No more "I took an arrow to the knee".

Models at around the 2B size aren't really capable enough to act a competent adversary - but they are great for something like bargaining with a shopkeeper, or some other role where natural language can let players do a bit more immersive roleplay.

[1] https://github.com/nobodywho-ooo/nobodywho

Could it be that these are just projects that use libcurl in some way?

I'm having trouble finding any sources that say that Daniel Stenberg actually worked on spotify, utorrent or openttd directly - just to test three of them.

A cursory glance at geekbench shows that you can get around 50% the performance of a 2020 Macbook Air M1, if you get the upgraded CPU option.[2][3] I have no idea how useful those benchmarks are, though.

[1] https://shop.mntre.com/products/mnt-reform [2] https://browser.geekbench.com/v4/cpu/16291496 [3] https://browser.geekbench.com/v6/cpu/2504172

While I think python is a really great language for advent of code, I'm not sure I'd recommend going for an OOP-heavy style. Although that might just be a matter of personal taste - I think OOP is a poor strategy for most problems.

Comments URL: https://news.ycombinator.com/item?id=26915944

Points: 6

# Comments: 1