I mean you expect some company which did some android rom, a smartphone, IoT Robots stuff a car?! and now makes a chip which can compete with apple.

The normal LLMStudio stuff works great, but then i tried out anything with subagent things or parallel stuff and it trashed my cache and got super sluggish/slowish.

What do you run and how?

I'm not an apple fanboy nor a xiamoi fanboy.

Its impressive that a random chinese company was able to catch up to the 2th richest company with global experts so fast and this achievement is not dimnished just because apple announced their M6 today.

Are you some fanboy?

Apple is the second richest company on the world (which doesn't need help/protection?!) and they have experts in chip design.

Xiamoi is some random chinese company not known for high end chips and was able to catch up impressivly in a short period of time.

This fact doesn't get funny or wahtever just because apple brought out a new chip today.

I'm not a fanboy for any of it and do not care.

I just wait for practical real live numbers with real stacks i use.

And btw. they reduced the memory bandwdith on the M4 in comparision to its previous version by 40gb/sec or so, you need to wait to see.

It is still slow, a lot slower than what you are used to with claude and co.

And as soon as you increase context size, your memory requirements jump.

Then when it runs for 30 minutes for something claude needs 5, your device will get hot.

And even a used 3090 is apparently now between 1-2k.

I'm very very surprised that Xiaomi matches Apples speed even with the newest release, its not diminishing Xiaomis success.

1-2$ / hour.

I'm not regretting my setup at home as it got paid by my company which makes sense here, but paying for electricity is quite high and makes already 0.3$/hour alone.

I would argue, the most interesting use case for running it at home is some personal agent which you want to run 24/7.

They will never ever do like this.

They for sure would add architecture ideas from other research/models but thats it.

How do people even become like 60-70 or 80 around me without all this magic?

tztztzt

But the agentic layer is being worked on, agents will start consuming more and more tokens

Even a year ago when i was trying to do a hugo template manually with the help of the documentatin /tutorial, it was shit. The LLM at that time, was better helping me than the documentation.