Show HN: I run 30B 22tok/s, 109tok/s not novel,6GB/16GB RAM overcoming llama.cpp

federicoTXTS — Wed, 29 Jul 2026 03:47:58 +0000

Democratisation of local AI is key. I've been working on pushing the limits of commercial hardware, squeezing any extra bit possible. My Scientific Agentic AI hareness helped me to reallocate every single bit of it. I rewrote the Kernel, I went down the CUDA rabbit hole until I have been able to explain any bit and any ms of computational power involved in the process pushing the Qwen 30B-A3B from 8 tok7s to 19 tok/s with llama.cpp up to 22.2 tok/s with my project and 109 tok/s on not novel content and speeding up the prefill by 5-9X

Comments URL: https://news.ycombinator.com/item?id=49093202

Points: 4

# Comments: 0

Show HN: Run GLM-4.5-Air(110B)on a 16GBRAM consumer machine

federicoTXTS — Thu, 23 Jul 2026 22:24:15 +0000

Article URL: https://github.com/FedericoTs/quantprobe

Comments URL: https://news.ycombinator.com/item?id=49028865

Points: 4

# Comments: 0

Hacker News: federicoTXTS

Show HN: I run 30B 22tok/s, 109tok/s not novel,6GB/16GB RAM overcoming llama.cpp

Show HN: Run GLM-4.5-Air(110B)on a 16GBRAM consumer machine