Keep in mind though that this is also sometimes use case dependent. Off the shelf implementations are generally pretty good overall, but can have pathological behavior on specific workload shapes you care about. So do this as a somewhat later optimization, and particularly when you see performance characteristics that don’t seem to make any sense.

1. Top end model on high/xhigh thinking (last time I did it it was Sol xhigh I think)

2. Make sure it creates some representative fixtures of different sizes and sets up a good testing, profiling and benchmarking loop that doesn't require my input.

3. Make sure it has access to reference implementation code

Edit: Oh and one obvious pitfall that for some reason I still have to remind even smart models of from time to time: make sure it knows not to try to parallelize its benchmark runs. I've occasionally had an agent struggle to figure out absolutely nonsensical data because it tried to run multiple tests on the same compute hardware simultaneously.

Everyone realizing that photos don't prove anything would defeat 100% of malicious users. I don't understand what people incorrectly trusting photos is supposed to achieve at this point, in your view.

You will not build a perfect system, or even something near perfect. The best you're going to do is make it so that it's hard to casually present AI photos as real, leaving only the cases where it really matters. In the "best" case, you've just made the public more trusting of photos in general, so that when there's actual money or power on the line that makes jumping through the hoops to fake authenticity worth it, the public is more susceptible.

The best outcome at this point is for everyone to get on the same page that photos have roughly the same probative value now as drawings. Poorly thought out snake oil efforts to prove authenticity are only going to delay that.

But where did you hear that they’re buying “all copies”? And to what end?

Aqua was garish and in your face, but it needed to be in order to be a symbol of what the system had achieved, which I think was especially important because that came at the cost of being a buggy, immature OS that had to run crucial apps in a hideous compatibility mode while it was getting established.

Edit: and if I can be a cranky old codger for a moment, I think that Liquid Glass unintentionally serves just as well as a symbol for what Apple has become: a company grasping to relive the memory of innovation without remembering how to innovate.

And to be clear, building intuitions that fail at certain scales is still a useful and important thing to do. But you haven't shown a scale where these intuitions fail. That's not insightful, it's just throwing smoke bombs for no reason.

And while the real situation at scale is more complicated, the math is going to come out to the same answer, albeit with extra terms muddying everything up.

If someone says that something true can be illustrated intuitively with a thought experiment, "sure, but what if we take that to a scale where our intuitions fail" is a sort of odd place to take the discussion unless you're genuinely curious how the math is going to shake out.

That is a very odd thing to say in a post that also mentions WebP. I would say that of all the commonly supported lossy image codecs released since 1992, JPEG takes the least advantage of human perception quirks.

KV caching is a super interesting engineering space, especially when you’re talking about local models where compute and memory bandwidth are highly constrained and you’re trying to trim fractions of a second everywhere you can by flipping between different ICL prefixes. But selling caches for specific documents just makes no sense at all.

When your abstract was clearly generated by an LLM and not curated to at least make it sound human, it does not make me want to read your paper.

So first off, I mean, of course you didn't, right? You had already learned most of what you were going to learn from this specific project by doing it by hand.

But yes, if you use AI to do projects that might have been at the edge of your abilities pre-AI, you will learn a lot less. The solution, I've found, is to get more ambitious until you're back to the edge of your abilities even with AI. You should be asking the agent questions like "has anyone tried?" and keep pushing until it isn't sure, and you're not sure you have any idea what you're doing. Then ask questions until you feel like you sort of know what you're doing, and verify your understanding by directing the agent to build.

And that's fine, right? Like people can do what they want. We're just going to get used to it.