I find forcing it to visualize things immensely helpful. I'm usually studying git diffs but when working of a big feature or refactor that can just be too hard.

I've never been very pro "visual programming" and always hated UML et al, but part of me is starting to wonder if it's time for us to give it another serious go.

what does 'minting' mean in this context?

Time unfortunately that you are simply not given because the whole point is to work faster so expectations of velocity have of course increased.

So you end up having to get a fuzzy idea of a given changeset, look at it a bit and evaluate quickly if it's a good change or a bad one and click it through, and on to the next thing.

At the size of PRs being whole features and doing this 5x faster than before, all you've got left as a senior is your Spidey sense. As a junior.. no idea how they cope.

We'd all love to still spend that 90% thinking time, but it used to be justified by necessity of getting the job done. Now, it's just not a luxury that is afforded.

What can ya do.. I'm slowly learning to be the best bot herder I can be, but it certainly feels like a change in skillsets. I definitely only feel like I'm able to do a good job due to enough experience and building and growing software projects over time to have an idea of what kind of decisions might bite us down the road, as well as to know what maybe can't be known but is just worth a risk. Without that kind of intuition I think I'd feel like I was really shooting blind.

Regardless of security and safety and other concerns, it just seems weird to me that OpenAI wouldn't be constantl monitoring these training runs for traces that are clearly going off task, and ending them. Because that just seems like it's going to be generating garbage training data.

Granted, detecting "off task" may not always be easy, but when they are literally writing out messages to each other overtly admitting that they are trying to find ways to fool the evaluator, I mean, even a regex filter could have caught some clues here.

In my reading, people aren't really saying "the AI is at fault", they are saying "hey look here's proof that this is dangerous". Like pointing at all the dead bodies caused by the virus and saying hey maybe we should stop making this virus.

The "moat" is data. And compute. But mostly data.

It is a classifier, that you don't have to train, that seems to work well, and they have made available by API.

One of the reasons for LLM success has not just been that they are "smart" but that they are smart while not requiring you to collect your own data and perform your own training.

There is also the marginal advantage of it not requiring a PhD to know how to use. But most companies can get a competent data scientist on board, so that is not a major problem, they could find someone to train their classifiers. But even a good data scientist can't make data appear out of nowhere. If you can get a pretty good result without needing data and training, for cheap, you're going to take it.

(Let's put aside the fact that almost every company using LLMs these days is irresponsibly NOT comparing the results with their own internal gold standard datasets for validation and calibration. They are just winging it. And Jev allows them to continue to do that.)

The idea is to drive actual testing from this, but in this era, I think it's interesting as a way to use AI to generate tests, and to cross-check those tests with the natural language descriptions, in a bit of a cycle that helps refine the highest level definition of the software.

Once that's nailed down, the implementation is just details.. normal engineering concerns like maintainability etc notwithstanding of course, but you can trust more and more the AI agents to get it right. The design specs being natural enough for humans to deal with but interpretable/specific enough to actually generate tests is pretty interesting for the bottleneck you are talking about, I think.

There were a couple of things in the article which i struggled with, this being one of them, and the other being:

> Quartz doesn’t rhyme with parts, but rather shorts.

This made me realize that in both these cases, I sometimes say it one way and sometimes the other. Probably depending on context. And probably because I'm Canadian.

How is this idea generally working out in comparison with Jev? I'm curious, from what I read so far it seems like Jev is still beating this kind of thing.

But it's curious because it's not entirely clear why, from an architecture point of view for all we know that's exactly what they're doing. So it must come down to the quality of those logits, ie., model size and training details.

It seems to me that what most of these single-token-prediction projects are missing is that Jev seems to be claiming they predict well-calibrated probabilities. This is an incredibly valuable thing that LLMs simply can't deliver unless they are trained specially for it.

As for "on accident" I've always assumed this was just a mistake from childhood, kids equating it with "on purpose" and never being corrected. A bit of a bone apple tea.

However I get your point, I used to correct an Indian colleague when he would say expressions a bit wrong, until I gave up and realized it had just become part of his local dialect in the region he was from. (Eg. "I have to go pick my kid" instead of "pick up my kid" which made me laugh at first and then drove me a bit crazy because he would consistently keep saying it.) At some point these things cross a line and become somehow "correct" again, that's just language for you.

A native English speakers we are in a particular spot where we have to just accept that the majority of the world is going to be speaking our language non natively, and that is fine once you get used to it.

People will often reach for "easily trainable, cheap" solutions when they can; the reason people reach for Transformers and LLMs is because when you throw more data at them, they get better.

I see these all the time here in the Netherlands. Very small cars. Some on the road and even smaller ones people drive on the bike paths. Small motorized vehicles, ebikes and scooters share the paths with regular bikes which works because they have usually at least two lanes, so they are more like tiny roads. Very handy.

On the other hand, I think this denies the reality (in my experience anyway but I think enough people will agree) that one often solves a problem as they are working on it.

This method seems to presume that a good engineer will submit a well thought-out solution or direction giving the AI an extremely good overview of each problem and enough of a description of what to do that it will do things as expected and they can just review the result.

In my experience it just doesn't work that way in practice. One learns the problem and even the domain while developing the solution. So one would have to submit at least a half developed solution not just "instructions", for there to even be coherent instructions in the first place. And one needs that experience working on the problem to be able to properly evaluate a separately proposed solution.

All in all for me this leads more towards using AI as a co-developer than using it to just implement some fully thought out idea and then check what it did.

The holy shit moment was partly learning about all the things that they did but if it was one or even 10 agents coordinating on something it would be, like, oh thats pretty amazing.

The actual "holy shit" for me is that this comes from a massive training run of all things, not an on-purpose, let's coordinate some agents to see what happens, but really just from a massively parallel set of individual agents that were supposed to be isolated.

That they spontaneously started doing this, and coordinating literally 10s of thousands of instances of themselves, is just.. mind blown.

That no one stopped it.. and that they actually did what they did.. is just a whole other level.

For me personally though it's not the fact that agents can coordinate so much as the massive scale at which it happened, and how this so obviously generalizes to what might happen if it were done on purpose.

I really see this as a stroke of luck, to be honest, that this happened in such an innocuous way. It resulted in a real hack, yes, but overall no one really got hurt and this is going to open a lot of eyes to what we should worry about going forward, in a geopolitical sense. I know it has mine.

As in, instead of all the complexities induced by discrete token generation, just generate the image of the text using standard image diffusion methods, then convert it to text.

If you used a single, monospace font, I bet this would be even pretty efficient, because the OCR problem becomes basically just direct template matching.

But I guess probably there is already a paper out there, I haven't searched. I'd be curious to know if it compares on par with token-based methods.

[0] https://en.wikipedia.org/wiki/Instrumental_convergence#Paper...