Like most projects, some indie productions (like TADC) end in creator burnout, others (like iron lung) end in moderate success, most probably end somewhere in between. We saw this with undertale, lights out, hazbin hotel, primer, el mariachi…

I guess today’s post represents a change.

I don’t have any public evidence to support my claim, sorry. Take it or leave it

Since Microsoft owns GitHub, their decision to shut down Windows 7/8 action runners is a sneaky way to help drive customers toward Windows 11.

Real-world systems need to be able to say "I don't know." This is a test about misinformation after all, and overconfident responses contribute to that.

Teasing out the difference between "avoid" and "unknown" could be a different research question

TL;DR Ed argues that the deal between Anthropic and xAI could have been negotiated in such a way as to make Anthropic only appear profitable during its “ramp-up” period in June, which incidentally is also the month that Anthropic is making tons of other pricing changes.

However, I've found that speculative decoders don't help much if you're running a model locally on limited hardware (for instance, my 32GB VRAM M1 Max from 2021). For one, you have to fit both the large and the small drafter model in memory. For another, if you're running a quantized model, the activation distribution is different enough that the draft model has a hard time guessing what's coming next.

My take is that speculative decoding is most useful on _very expensive_ prosumer/hobbyist setups where you have 128GB of VRAM and are running your local models with full fidelity. It's also helpful for inference providers where they can send output tokens at a computational cost slightly higher than their input token cost.

When Claude acts up, my strategy is to rewind the conversation to the point where the misunderstanding started, revise what I said just before, and then continue from there. I’ve found that letting one mistake enter the conversation seems to make further mistakes more likely.

The user already has omnipotent power over the agent’s sense of time and memory. We can rewrite what Claude sees and hears. Can’t do that with humans, but rewinding such an important function in Claude that it has a top-level keystroke.

Why spend time, tokens, and cortisol arguing and demanding the pet rock step through an apology protocol?

If you don’t read the tests to check they confer your intent or specifications, they’re more like tautologies than tests, you know?

LM-Studio doesn't support certain parameter combinations. For instance, LM-Studio supports KV quantization....but if you're using the MLX backend, you can't set the context length when KV quantization is used? Why? Running a model with certain settings requires keeping a little SAT solver going in your head. I found that overwhelming, so I just stopped using it.

The Ollama devs want to offer a central curated experience, but I perceive their approach as "playing fast and loose." They've re-implemented unique code for every model they support in their own Go runtime, so certain parameter choices aren't supported. On my hardware, their MLX backend just doesn't work at all without segfaulting the server process for example. It doesn't smack as vibe coded the way oMLX does, but it also doesn't smack as professional or battle-tested.

Ultimately, just dropping down to llama-cpp's GGUF model support and asking for default settings has provided faster inference speeds than anything I've been able to benchmark with them, but everything's within 10% of each other anyway so it's not a huge deal for me.