I took its preliminary findings into Claude Code with the same model. But in mine it knows where every adjacent system is, the entire git history, deployment history, and state of the feature flags. So instead of pointing at a vague problem, it knew which flag had been flipped in a different service, see how it changed behavior, and how, if the flag was flipped in prod, it'd make the service under testing cry, and which code change to make to make sure it works both ways.

It's not as if a modern Opus is a small model: Just a stronger scaffold, along with more CLI tools available in the context.

The issue here in the security testing is to know exactly what was visible, and how much it failed, because it makes a huge difference. A middling chess player can find amazing combinations at a good speed when playing puzzle rush: You are handed a position where you know a decisive combination exist, and that it works. The same combination, however, might be really hard to find over the board, because in a typical chess game, it's rare for those combinations to exist, and the energy needed to thoroughly check for them, and calculate all the way through every possible thing. This is why chess grandmasters would consider just being able to see the computer score for a position to be massive cheating: Just knowing when the last move was a blunder would be a decisive advantage.

When we ask a cheap model to look for a vulnerability with the right context to actually find it, we are already priming it, vs asking to find one when there's nothing.

Given that it's absolutely impossible to stop people not aligned with us (for any definition of us) from doing AI research, the most reasonable way forward is to dedicate compute resources to the frontier, and to automatically send reasonable disclosures to major projects. It could in itself be a pretty reasonable product. Just like you pay for dubious security scans and publish that you are making them, an LLM company could offer actually expensive security reviews with a preview model, and charge accordingly.

There's a practical difference to how much better certain kinds of results can be. We already see coding harnesses offloading simple things to simpler models because they are accurate enough. Other things dropped straight to normal programs, because they are that much more efficient than letting the LLM do all the things.

There will always be problems where money is basically irrelevant, and a model that costs tens of thousand dollars of compute per answer is seen as a great investment, but as long as there's a big price difference, in most questions, price and time to results are key features that cannot be ignored.

The PRs that it comes with are rarely even remotely controversial, shrink the codebase, and are likely saving tokens in the end when working on a real feature, because there's less to read, and it's more boring. Some patterns are so common you can just write them down, and throw them at different repos/sections of a monorepo. It's the equivalent of linting, but at a larger scale. Make the language hesitant enough, and it won't just be a steamroller either, and mostly fix egregrious things.

But again, this is the opposite of the "vibe coding" idea, where a feature appears from thin air. Vibe Linting, I guess.

As usual, blame the suburbs, which make all kinds of infrastructure quite a bit more expensive per capita.

You will find new houses that small, but typically when it's extremely high value land, so typically infill. And then chances are it's a multi story house that fits the lot to the limit.

So of course we price businesses based on the expected long term value of the shares, as best as we can guess it. But the fact that a company degrades in value as it "overgrows", and engorges itself to become an entity that can't innovate or do anything efficiently in itself goes into the price too. It's not as if a place like IBM doens't want to grow: We just know they won't.

As for speculation rather than dividends, I suspect the real medium why this happens isn't just need for infinite growth: Again, as growth expectations slow down, price moderates: See Paypal vs Stripe. The issue is mroe of a principal-agent situation, as it's very difficult for the median shareholder to, say, force Zuck to stop spending money on the metaverse. And it's not just at the top level: We have a lot of incentives in organizations for people to push for more hires, even when there's very little value to be had. Anyone with a long career can see how much less tense a growing company is that one that has decided its headcount is stuck for a long time, or possibly shrinking.

Principal Agent problems are just much more annoying to put a blame on, because instead of being able to blame some exec all on their own, we get to look at ourselves too, and how what is good for us differs so much from what is good for employers too. The blame is spread thinly, and the behaviors that would lead to more efficient companies are also worse for workers. Then it's suddenly people easier to like, and we don't like where "try to be profitable at the most optimal size" takes us.

One thing I recently did was run a pass over some unit test and functional test suites, asking for standardization on initialization, and creating reasonable helper methods to minimize boilerplate. Any dev can do that, if they have a week, and it'll future code changes more pleasant later. For Claude, an hour was a -8000 line PR that kept all the tests, with all the assertions.

It's what people need to figure out out of a a codebase. Our normal quality practices have an embedded max safe speed for changes without losing stability. If you use LLMs to try to change things faster, the quality practices have to improve if one wants to keep the number of issues per week constant. Whether it's improving testing, or sending the LLM to look at logs and find the bugs faster, one needs to increase the quality budget.

There are all kinds of bad externalities caused by seas of asphalt that is unused 95% of the time, but few countries are all that interested in using any mechanism to make the property owner pay for them.

But is this something that is best done top to bottom, with a big report, counting tokens? Hell no. This is something that is better found, and tackled at the team level. But execs in many places like easy, visible metrics, whether they are actually helping or not. And that's how you find people playing JIRA games and such. My worse example was a VP has decided that looking at the burndown charts from each team under them, and using their shape as a reasonable metric is a good idea.

It's all natural signs of a total lack of trust, and thinking you can solve all of this from the top.

Most of the costs are ultimately salaries to Americans, and money handed to American companies, so most savings would come from someone's livelihood. That's why we cannot reform: The party that actually cuts costs will build resentment for decades, and create a blip of unemployment. Nobody wants to do that, and therefore you aren't going to be a serious, relentless attempt at cutting costs. We've seen how the attempts that the ACA made were counteracted by consolidation at all levels.

Serious cuts have to have no mother. Say, if we ever did have an AI that worked well enough at this, and outcompeted primary care physicians. Foreign pharmacies bypassing all controls and being able to hand you much discounted drugs the day after. Telemedicine and cheap travel put together to make surgery that didn't involve an ER visit just as easy and much cheaper than using the US system. Straight out disruption, because the incentives are such we sure aren't getting improvements in regulation.

I was asked to look at the site when it was already live, and some VP of the parent company decided to visit the site from their phone at home.

AI, if anything, is amazing at collaborating. It's not perfectly aligned, but you sure can get it to tell you when your idea is unsound, all while having lessened principal-agent issues. Anything we can do to minimize the number of people that need to align towards a goal, the more effectively we can build, precisely due to the difficulties of marshalling large numbers of people. If a team of 4 can do the same as a team of 10, you should always pick the team of 4, even if they are more expensive put together than the 10.

Ultimately the American parent is paying for the kids education either way: Either by buying a more expensive house near said "good schools", or by paying a private school, which is allowed to be selective in their admissions and match students. Making all schools actually be about the same is not just a matter of funding them equally, but you'd have to end the student segregation (even when it's in legal ways(, which is quite the challenge.

For instance, around me, there's some really bad school districts that end up grabbing very large mansions. But what happens there is that none of the kids of people living in those mansions actually go to public school. So while it might not be economically difficult to up the funding of the schools near poorer neighborhoods, I don't even necessarily think that they will get the same outcomes for the same funding: The selection component is going to change performance.

Very different from the typical weekly/montly outage meeting, where discussion is actually expected, instead of being a ritual.

Now, does this mean it's the right way to talk everywhere? Of course not. And since it's often seen as safe, it's overused. But it doesn't just arise, as a bug. plain language that means what it says creates more conflict, and isn't always better.