> Where timing gets interesting is that forge will slow down workflows because the retries mean you don't error right away. Bare runs were failing fast in my experience. But on a per-call basis there's very little overhead.

> I haven't detailed it simply because the order of magnitude of a single LLM call is so much higher than all the overhead put together.

Yeah, that makes sense and seems fair. The sort of delays are almost and inevitability, you're not trying to improve speed, but by improving reliability, it can obviously increase overall throughput.

Having watched the demo video too now, automating retries etc would be helpful for me. It's impressive to see how quick the models run on better hardware, and the performance improvements are impressive, even if the overall run takes longer sometimes because it does more correct things. Thanks again!

I also agree that if I spent more time using cloud based LLMs, I would very much find local LLMs less capable and useful. Comparison is the thief of joy though, and I'd rather feel blissfully ignorance towards SOTA LLMs rather than a dependence on them.

Before taking a local focus approach, LLMs increasingly left me feeling a mixture of FOMO, sadness and futility towards the future of software and tech. I assume it's 100% a me problem, but it has it's benefits:)

I have a laptop with a broken screen and an RTX2060 at my disposal. I can run 12b - 14b dense usably, just, although I think 4b - 8b dense models give me the best tradeoff of speed and usefulness.

Larger MOE models with more parameters (20b+) but fewer active (2 - 3b) are sometimes a little bit slower, but are often far more capable.

For original trackpoints, it's basically a stick in the middle of an up/down/left/right resistive strain gauge.

For the ploopy beans here, they use hall effect sensors instead of resistive strain to get a bit more movement.

As soon as you have non-binary up/down/left/right values, the mouse direction and speed can be interpolated to so many values that mousekey accidental squares become impossible.

I've only skimmed the post and read the abstract and in some places you make a nod to how simple tweaks can make something 10x faster/slower, but then all of your metrics and data seem to focus 100% on accuracy. You need to address speed.

Specifically for agentic workflows and local models, accuracy around function/tool calling hasn't been a problem for me now for about 6 - 12 months, personally, since around QwenCoder3. The main issue is context management and the impact on timing, since agents will often swap prompts and break prompt caching and similar timing improvements.

It looks like your work adds a layers and wrappers like guard rails and retries. This would make my local model experience - specifically for agents - unusable because of the delays it would add.

I really appreciate and respect the work you've done, and apologies if you have already addressed this head on, but with so little talk about the impact on timing here, I feel like you're hiding something or overinflating the actual real world improvements here - what are your thoughts?

It's also mildly concerning me that nobody else has raised this - am I doing something wrong here, or is everyone else just not actually using local models in real life?! Talk to me about your speed experiences!

Someone is just asking to get Google's side and explaining why they want that, which seems reasonable since we're in a post where Google is being punched/blamed for this, and it sounds like it isn't Railways first questionable outage.

That's a very, very weird take on many, many levels. Could you elaborate a bit about where that view came from, how often you use AI, what's your career etc.?

I felt most of the extra functionality and polish that I guess makes up the massive costs of UHK and ZSA wasn't actually necessary. It was cool and fun and useful to try a bunch of different stuff, but then over time, I wanted things to be simple and small which UHK and ZSA Moonlander aren't (ZSA voyager wasn't at the time).

All I'm saying is if you've got comfortable with a cheap Corne, I think you might feel underwhelmed if you spend a lot on something a lot fancier.

But, just wanted to share that I was similarly surprised to land on mouse keys as a preference. I tried most of the UHK modules which were also pretty good and have since tried various other trackballs and pads, but since trying UHK mouse keys, they're what I keep coming back to most, even since switching to new keyboards.

One issue I have with mouse keys is fear of using them in front of others though: every so often, if I need to click something particularly small and don't have a keyboard shortcut memorised (vscode panel resizing is one) it can sometimes take me a fair few embarrassing seconds drawing small squares around my target before I resort to actual mouse hardware.

For the amount of time and thought and effort people have put into alternative mice, I feel mouse keys are massively overlooked and probably have a lot of room for software/firmware innovation without hardware costs.

I find Gemmas really good for a short conversation with maybe 3 or 4 exchanges of a few paragraphs each, which covers a surprisingly large amount of interactions.

For anything longer form though, particularly with larger code contexts, Qwen is far more useful for me personally.

I'm not an expert in this field, but my understanding is Qwen are hybrid gated attention mechanisms, whereas Gemma is hybrid including a sliding attention attention mechanism which makes it look like it favour the most recent tokens a little too much at times.

This is all in the context of local quantized models, I'm aware both have larger cloud variants that wouldn't suffer as much.

If you're sincerely trying to engage in good faith, I feel you should be apologising for your role in sending it in the wrong direction unintentionally.

To be clear, I'm not taking a position in the debate here, just commenting that the way your engaging is legitimately a bit annoying if you're not aware. The other person getting really angry isn't the best look either, but I'm sure they already know that.

It's obvious to you, me and the other 2 presumably techie people who've responded within 15 mins that you shouldn't have been using Kubernetes. But you probably work in a company of full of techie people, who ended up using Kubernetes.

We have HN, an environment full of techie people here who immediately recognise not to use k8s in 99% of cases, yet in actually paid professional environments, in 99% of cases, the same techie people will tolerate, support and converge on the idea they should use k8s.

I feel like there's an element of the emperors new clothes here.

I'm annoyed at myself, because I thought/hoped/praised chinese AI when they were opening up as Llama was closing, but Qwen looks to be doing the same playbook here as Llama/Meta, Gemma/Google and OpenAI/gpt-oss.

There are very, very few things that can be completely transparent without giving competitors an advantage. The nice solution solution to this is to be better and faster than your competitors, but sometimes it's easier just to remove transparency.

Most writing on juggling talks about moves and repetition. You'll see the same terminology and approaches everwhere, site swap notation, cascades, messes, 1 hand, 2 hand, claws etc.

This can sometimes give the impression that when you're juggling, you're basically doing a move, or repeating a motion, or following a script, like being able to perform the specific move is the skill and achievement itself, but it's really, really not.

I think it's better to think of the balls as instruments. When you're reading and learning about juggling moves like cascades and messes and clawing, you're reading about chords on that instrument. If all you know is how to play individual chords, and the names of the chords and theoretical and technical ways to represent the chords, you're missing the point of knowing how to play an instrument.

Most of the time when I'm juggling, I'm in the same creative and expressive mindset and headspace as I am if I'm soloing on drums. Every move is a different move between a bunch of different moves based on wherever my mood takes me. It feels good and allegedly look impressive too.

I've tried to learn keyboard/piano so, so, so many times and it just doesn't stick with me, I struggle to retain the theory, I can't get the intuition, I never find myself in the zone after years and years of attempts and lessons. Drumming was the opposite though, I felt comfortable on it within months of having my kit with very, very little theoretical background. I consider myself a theoretical person, but juggling - like drumming - seems to be one of those few things for me where the theory makes it worse/harder/less intuitive/less fun for me.

The sentiment still applies the parent comment of yours though.

That link doesn't have much affiliation with Qwen or anyone who produces/trained the Qwen models. That doesn't mean it's not good or safe, but it seems quite subjective to suggest it's the latest latest or greatest Qwen iteration.

I can see huggingface turning into the same poisoned watering-hole as NPM if people fall into the same habits of dropping links and context like that.