We run an agent company and outside coding the new Gemini 3.6 Flash and GPT 5.6 Luna are very interesting. Luna can do a bit of research and create reports. Gemini is great for computer use.

For programming it's all Kimi K3 now.

But you are correct, there's not enough swimmable lakes in Berlin and when the weather is nice you better drive somewhere in Brandenburg if you don't want to be stuck in a fully packed pond with 1000 others.

What a time to be alive.

Paid 20 euros in total.

https://www.home-assistant.io/voice-pe/

Then you can set the background AI to be any OpenAI compatible API. So I just created one to my local Rust Agent, and connected it to home assistant. Now I can yell from the couch to create me a new Proxmox container with the next free static IP address etc. :D

I can do things like take a photo of a doctor's note among add the appointment to my calendar, send a PDF to my archive tagged, OCR'd etc, search info from internet, look data from Google Maps.

I can even integrate this to home assistant and talk to my agent with my open source Alexa-like system.

I'm definitely having full eval suite on already if they get overloaded later on.

But yes, I also think it's not the greatest model for programming. On the other hand, for agentic tasks that are not programming related it's hard to beat Opus 4.8. It can try different things and pivot even when the user is not great with prompting. 5.0 seems to not be worse, but definitely wastes more tokens and costs more.

But $200/day is very common in the business. The monthly plans are just a trial version of what's going to come for all of us. That's why I am very carefully looking into the open models today.

Not very good for programming though.

Now if you look at where GLM stands, or even DeepSeek v4 Flash, things get really interesting for what they provide.

US labs completely miss a cheap model that can solve problems for 95% of the people. Gemini 3.5 Flash could've been it if it didn't burn so many tokens.

The eval is an agent that runs a set of tools and a prompt we can tune separately for different models. The OpenAI version of the prompt was specifically tuned based on their guide[0]. Then we let Opus to run another agent that acts as a user, trying to solve a problem (anonymized and taken from production). The problem is complex and we don't expect it to be solved by these agents, but we measure how the agents operate when faced with a vague problem:

- Opus 4.8 and GLM 5.2 both identified a constraint sooner and stopped so the user can fix an issue first that the agent cannot solve.

- Sol tried hard to solve the issue with different tools, burning tokens, until finally reached to the same conclusion with Opus and GLM. It was two times more expensive compared to Opus and six times more expensive to GLM for this task.

- Terra went even further and started calling tools that would not solve the issue, burning tokens and failing.

- Luna repeated the same failing tool call until it hit the round limit, and burned more money than Opus.

I'm kind of puzzled with the new GPT. Like, yes Sol is OK for programming, but I was expecting to get a cheap agentic model for non-programming tasks, one that can detect if things go awry and correct. Terra is too expensive and Luna not really fit for the task. Sonnet 5 is a bit better but more expensive than Opus 4.8, which is still the best in my evals. GLM 5.2 is extremely good if you can define the task and the tools clearly for it, and costs pennies!

[0] https://developers.openai.com/api/docs/guides/latest-model