Fable is doing - so far - a great job. I just had one big question around how part of it should work. I had a design sketch, but with some big unknowns. I asked fable to figure it out via reasoning and prototyping, and it did - it even, under its own initiative, wrote a fuzzer for its prototype which explored and verified that its reasoning was correct. It absolutely nailed it. And it found, and fixed, a couple bugs that I'd missed.

I'm sure its weaknesses will become apparent in time. But, wow this thing is a beast. Its the first time I'm reading the work of an LLM without spotting obvious weaknesses in its reasoning and code. I'm really impressed.

I want Claude and OpenAI to be able to compete with Siri on an equal footing. Just like Apple maps has to compete with google maps. Competition pushes companies to make better products.

Yeah, I'm going to act as a human claude frontend. But I don't think we're that far off.

the site is not much more complex than a “basic info site”. There are a few small dynamic elements - like a list of donors populated dynamically and a box to sign up to the mailing list. Sites like this are trivially easy for Claude. I got claude to do a mockup based on a screenshot. Its 5 minute mockup has a much nicer design and layout than the current website. 1 Claude code prompt later and it was functional.

But yeah, short term I’ll see if Claude can just update Wordpress directly. I don’t want to spend my weekends learning wordpress. But if Claude can do it for me? Great.

People are increasingly associating “AI art” with cheap slop. I wonder if the same will ever happen to programming.

I think there's still a lot more custom work than you'd expect. A recent example: My mum is ex-president of a small international NGO. When I visited earlier today, she was bemoaning how the lady who runs their NGO's website charges them an arm and a leg for small changes. As a result, the website is constantly out of date. The current site is entirely built on top of wordpress. There is, of course, politics. This lady holds the login credentials to everything. And she holds them close to her chest.

I showed my mum what we could do with claude design. Claude whipped up a much better looking, working version of (part) of the website in about 5 minutes. I think LLMs aren't quite at the point that my mum could manage the website herself. You do need a little technical knowledge. My mum just doesn't know what to ask for. But we're really close. I've offered to be a human frontend to LLMs for them going forward, if she can wrest control from the "webmaster".

I suspect conversations like this are happening everywhere at the moment. There are an incredible number of small websites out there. Most of them, claude or chatgpt could reimplement in 5 minutes tops.

Give it some time. We’ll figure out what LLMs are good and bad at. I think vibe engineering will eventually go up on the wall next to static vs dynamic typing and vi vs eMacs.

At least, that assumes AI models won’t keep improving by leaps and bounds. We’re in a transition period. It’s gonna be chaos.

Its weird, working with LLMs. There are some things the LLMs are extremely good at doing autonomously. Like, reverse engineering, or reading documentation (and using that knowledge in other areas). There are things that they can do - but you need to explicitly prompt. Like, I've found Opus is quite good at optimising code. I've had a lot of success by asking it to write benchmarks (and do profiling), and use that data to improve the performance of some piece of code. Thats often enough to get quite large performance improvements. You can get even further by showing it similar code others have written which is well optimised. It's very good at copying optimisation ideas from one project to another.

But then there are very simple things it really struggles to do. Some kinds of correctness testing. Invariants. System design.

Is it bad at that stuff, or do I just need to figure out how to prompt the LLM? To return to the topic of this thread, I think we're seeing a lot of different opinions on LLM generated code for 3 reasons:

1. Some people aren't looking at claude's output at all. Some people are looking at the code and it looks fine to them. And some people (with more experience writing software) are looking at the code and judging it poorly.

2. We all prompt our LLMs very differently! It turns out that you get really different results based on how you prompt the machine. We're all figuring this out together. Some people have better instincts than others.

3. We're working on different projects. Claude is comparatively much better at end-user facing software. Its great at making a standalone website. Its much less good at finding and fixing obscure bugs in large, established pieces of software. If you work in consulting, LLMs can already do a lot of your job. If you work on Chrome or Unreal or the windows kernel, maybe not so much.

It’s also strangely bad at correctness. You can ask it to write unit tests for a project. Unless you’re careful with your prompting, it will only write the unit tests that it knows will pass.

Generally, these models are amazing tools. But they cannot be trusted to make correct, maintainable software. At least not yet. Maybe in another year or two.

Apparently there were protests in NY of all places on our behalf. I don’t know what they were hoping would happen - would the state of NY ask our state to change our laws for them? How bizarre. Our local policies are up to us, thanks.

Surely NY had other things to worry about at the time? The news we were hearing of ambulances in NY queueing outside overpacked hospitals… though I suppose the media there was saying equally scary things about life in Melbourne.

Our lockdowns didn’t work, but we loved our state premier for trying. He was so popular that the following election, the other political parties didn’t really bother to show up. The opinion of New Yorkers was against the will of most locals here. It was sweet to protest for us. But it had very weird vibes.

Heh it’s a lot more than that. About 1/3rd of my Australian friends have a story to tell about unfortunate US border crossing experiences. I personally know two (white) people who were denied entry at the border - in both cases for allegedly ridiculous reasons.

My partner was arrested at the beach once in the US. The police wanted her to narc on someone she was travelling with and she refused. (The case was thrown out of court by a furious judge, but it was a whole thing).

Some of ICE’s detainees may have different opinions on that point.

The UK may endow her citizens with fewer rights. But I have a lot more trust in British due process. British civil servants seem much less … capricious than Americans.

I was almost denied entry to Hawaii once because I told the CBP agent I didn’t have any cash on me. (My money is in a bank account, obviously). He went on a big rant about how expensive Hawaii is. I think he was worried I’d end up homeless. (Even though my visit to hang out with my then employer.) Over the years I’ve heard so many stories from other Australian friends about wild and unfortunate encounters with US police and officials.

By comparison, the British government seems far more civilised. If something happened while visiting the UK, I have much more confidence that everything would be resolved in a fair and reasonable manner.

I‘m also careful with what I say online. US CBP tracks what people post online, and has been known to deny entry to people for being critical critical of the current president. I don’t want to risk losing out on future opportunities in the land of the free.

As a solo operator, autofocus is great. Maybe the right metric is the number of crew per camera. 2-3 crew per camera? Manual focus is fine. 2-3 cameras per crew member, like solo filming a podcast or a theatre show? I'd choose autofocus every time.

Its probably a scale thing. Photography lenses make up for the design, engineering and manufacturing costs with scale. Everyone who takes photos needs lenses. But far fewer films are made, and cine lenses are often rented. So they really can't be manufacturing that many units in total. I suspect they don't manufacture cine lenses in high enough volume to justify the engineering costs of fitting complex microcontrollers and motors into the lens housing. And if the production can afford to hire a focus puller anyway, autofocus just isn't that valuable.

For some reason, cinema lenses are still - for the most part - purely mechanical. For film and TV, most camera operators still focus manually - often via gears attached externally to the lens.

Coming from modern photography, manual focusing is inconvenient and difficult to learn. But there's something very old-school cool about cine lenses. They feel great.

Its also really bad at inventing and leaning on invariants. I make rules in my code all the time - "by the time we get to path X, we know Y and Z are true.". In aggregate, these invariants make code simpler and easier to reason about. But claude doesn't do that. It just kind of - slops through and adds bespoke "just in case" workarounds all over the place. Every time I read through code its written - without fail - I find bad design / architectural choices.

Maybe mythos will change this. But for now I've slowed way down on my claude code usage. You can't build a skyscraper on a foundation of mud.

> then wouldn't they also optimize the naive piece of code?

Great question. They do sometimes!

In general, the problem for compilers is that its not obvious which method would be better in some given piece of code. Most branches are highly predictable. Like, imagine a for loop which counts to 1000. At the end of the loop body, the code branches to see whether we should stay in the loop, or exit the loop. The first 999 times through the loop we keep going - so 99.9% of the time, the branch ends up taking the same path. Its very predictable! CPU designers optimise heavily for this, via branch prediction logic. Highly predictable branches run fast. (This is also why array bounds checking doesn't really hurt performance at all.)

But the branch predictor really struggles when the condition is unpredictable - ie, when a conditional branch is taken about 50% of the time. As is the case in a sorting algorithm.

The compiler has no idea whether any condition in your code is predictable or not. There are hints you can use, but it often defaults to just doing whatever you ask it to do.

Here's what the compiler actually does with the code you quoted. You can see the extra branch + jump for the second version of the code:

https://c.godbolt.org/z/zv7Tcd49f

I clicked around - for some reason even using __builtin_expect_with_probability, none of the compilers I tried will convert from one version of this code into the other.