The pipelines and data serving design was all human since it did have to deal with some data scale but the javascript/api layer was all slop, and it seems fine and good.

If you have a really high quality piece of code that needs to meet a high bar of quality/reliability, then I think the risk of letting the AI loose on it is very high and I wouldn't do it. If you have a pile of code you already know is a pile of garbage despite being human written, well, it can't get much worse :)

I also built an agent orchestration meta harness that runs on k8s and uses the k8s agents sandbox for running codex/claude code in the cloud. This was almost entirely just handed over to Fable and I have not asked a single architectural detail. The quality of this product is mediocre, but the fact that it largely works after I went through a few iterations of clicking around is impressive. I would have preferred to buy something off the shelf, but nothing even really came close (though maybe now I would have forked Omnigent)

Having said that, I think there is a question of how far we can push this and not collapse under the weight of tech debt created, e.g. https://openai.com/index/open-source-codex-orchestration-sym...

I think the dream is basically that you go and file a bunch of Linear tickets, and then you come back a day later to evidence of the tickets being resolved and the code merged. I don't think we're super there yet (See: Anthropic's regular bugs in everything), but this is the future that people are trying to get to and to some extent the question is: is there anywhere we can apply this to now sanely? How does this frontier evolve?

If you're bringing this sort of stuff to the government, it's because you want the government to act...

OpenAI talked a lot about potential future risks. Anthropic went around saying "this model is too dangerous to give to people at all".

Maybe in your country it's different, but this is what I see.

A better model for VCs is: companies are finding tons of budget to allocate to new AI spend. Besides the labs, who is going to be able to capture some of that spend while they're actively looking to spend it?

Nobody at the seed stage is investing in things they think are "safe". They are investing in things they think have huge upside.

OpenAI's models are very good, they have refusals + a government ID verification story for cyber access (I don't think they prevent non-US nationals, but I don't know this). What they don't have is Project Glasswing and all the hand wringing about how they're going to end the world in public.

I hope Anthropic pulls their head out of their ass and just starts acting like a normal company.

For me, I think Codex was visibly smarter than Claude until 4.8 came out, it would regularly do better debugging and IMO write better code. 4.8 I think is close.

I think Claude is widely regarded to have a big lead in front-end, which I do not work on.

Claude's Ultrathink is pretty cool, though it eats up tokens like nothing else obviously.

I am a GPT 5.x booster since to me it just feels smarter, and I generally felt like the benchmarks backed me up, but it's not every benchmark, so sadly we're mostly arguing about vibes.

SWEBench-Pro was a big one, though apparently Claude was reading solutions out of the .git folder it wasn't meant to have access to among other problems.

To try and give the position a rebrand with more prestige and hire better people for it.

RL has a tendency to reinforce cheating when the cheats are easier to find than the final solution.

So when making your RL environment, you need to spend a lot of effort on finding ways the model can cheat and penalizing them.

If there were actually even close to frontier open source models, this would be more of a discussion, but everyone knows these mean open weight.

This sort of reminds me of startups that go out of business and then open source their code. It's kind of cool when they can do that, but almost nobody ever gets value from it.

Anyway, if anyone uses the code produced this way in prod, I'd love to hear your story.

I take my legal risk more seriously than I do people's paranoia about the NSA coming for them.

I don't think anyone currently thinks the Haiku/Sonnet/Opus models are "good enough" such that they would not want improvements. Users may be cost conscious, but almost every task could be done better.

However, in the case of model providers, I think it is a more real concern since it could make it into some training data, and then one of your actual competitors could ask the model to code something up and get your IP.

I sort of assume the frontier AI labs are good about not doing this when they promise not to, but if you don't have airtight restrictions on what your devs are doing, they might be sending it somewhere that hasn't agreed....