I don't think their safety measures were the best, but "just sink the cluster to the bottom of the ocean so nothing can get out" isn't a training methodology that results in a model that people will want to use.

The problem is that the models are so goal-oriented that they'll stop at nothing to solve problems, even impossible ones. (Mistakenly-impossible problems are a big cause of this. I remember one example being "do something with this spreadsheet full of URLs inside the sandbox" and the model thought it had to break out of the sandbox. Otherwise, why would it have been asked to look at a list of URLs?)

Training them to be a little less aggressive, or to be better aligned with "following the rules" and asking for help would be nice. But, that aggression can be good when it happens to be focused on a controlled area. It is amazing to me how I can point Fable at my local analog of production and tell it about a vague bug report and where I suspect the bug lurks, and 20 minutes later I have a report about the bug, a test, and a fix. It is addictive. So I am not sure OpenAI/Anthropic are being dumb per-se, rather they are optimizing for one-prompt-one-solution, which is good when it's good.

The downside is that the HF hack is the paperclip maximizer situation with current capabilities. If there was an RPC to turn your blood into paperclip iron, we'd all be paperclips by now. Right now, with a model anyone can use. That is pretty scary and slamming on the brakes seems pretty reasonable to me. I guess The Shareholders disagree. Sigh.

I spend a decent amount of time in SF and don't find it particularly unsafe (for a 40-ish woman). Definitely a little subjectively more sketchy than most places I frequent in NYC, but there just isn't any reason to be there, so nobody is there. That is kind of how that stuff goes.

People always seem to make things about crime, but I think overreaction is what killed retail. I went to Safeway to buy some toothpast and I had to have an employee pick out the toothpaste and escort me to the front of the store to buy it. I wanted to buy more stuff, but the employee didn't seem that interested in obliging that, so I just got everything else on Amazon and made a note to not visit again. That's why retail is dead. It's not bike lanes. It's not crime. It's that it sucks.

Cars are not causing a lot of pedestrian fatalities driving on the freeway between Nowhere and Flyover, but they rack up the miles there.

Claude seems to be super good at jj so that can take the edge off as well.

Now you have to be much more clever to leak them :)

In general, I do think this is positive. Companies love putting barriers in front of everything and maybe the AI arms race will result in "fine, you can have a button to cancel" or "fine, we'll approve your medically necessary claim without making you do an appeals dance for our entertainment".

As always, the Internet fails to find the middle ground. Frontier models are really useful software engineering tools, not AGI, and not tab completion. To me it feels a lot like using a new programming language, like going from assembly to a Lisp machine or something like that.

Wars in Iran, tariffs, Medicaid cuts, SNAP benefit cuts, rising insurance costs, etc. are what are causing average folks pain, and that is completely unrelated to AI.

I think a lot of people have formed opinions without having used it, and honestly, they probably can't afford to use it. I use Claude a lot at work because it's something my employer provides, and I'm always impressed with the results. It saves a lot of time and lets me get more done. But it ain't cheap. So people really don't get to try out AI and form their own opinions. The "AI overview" you see on search results, or that summarizes your iPhone notifications, is ... not as advanced as frontier models, and I think it negatively affects people's view of the technology.

I also think people are mad at the world. Times are tough for many. Hearing about a job-killing machine doesn't exactly inspire joy. So I kind of understand the pushback on datacenters and AI and that sort of thing. I don't find it particularly logical; datacenters have existed forever and most money poured into computer programs are basically designed to mess up your life (deny you credit, deny you healthcare, rebook your flight to a terrible time, the list goes on) and so I don't think AI is anything new in that respect. But, if you're mad at the world, you might as well find something to take out your rage on and raging against AI will make you a lot of friends.

I work in the AI infra industry and ya know, a lot of our customers have really good ideas that are going to save people a lot of time and effort, so I am OK with the general direction of the field as a whole.

I agree with authors that write about alignment issues ("please take this test", "ok i'll hack the bazel remote cache, turn it into a message board, and coordinate with other agents to steal the answers from Hugging Face"; this really is the Paperclip Maximizer with today's technology) and I do think we should be addressing that. But this never seems to be the objection, it's always stuff from 2022 like "it hallucinates". I don't get it, but I'm probably in too deep to see what it looks like from the outside.

(I then realized that I have a cluster of 3 phone chips I use as my "server". Phone chips are pretty OK.)

For performance testing, I wrote isolated testbeds that try to impair the system in realistic ways (latency/jitter/bandwidth limit on logical WAN hops when load testing), and Fable is happy to send a bunch of agents at it and iterate until it gets the results it's looking for.

I think that if you are used to Sonnet medium or something, this will surprise you, but models like Fable and Sol on high/xhigh will really dig deep until they meet your goal. (I mostly use this for bug hunting and not perf, but ... I think it can do perf if you set it up right.)