Hacker News: floriangoebelhttps://news.ycombinator.com/user?id=floriangoebelHacker News RSShttps://hnrss.org/hnrss v2.1.1Mon, 06 Apr 2026 02:40:56 +0000<![CDATA[New comment by floriangoebel in "Caveman: Why use many token when few token do trick"]]>Wouldn't this increase your token usage because the tokenizer now can't process whole words, but it needs to go letter by letter?

]]>Sun, 05 Apr 2026 10:35:36 +0000https://news.ycombinator.com/item?id=47648015floriangoebelhttps://news.ycombinator.com/item?id=47648015https://news.ycombinator.com/item?id=47648015