We don’t see electricity companies among the biggest companies in the world, except in China where they are a state owned monopoly… maybe a good analogy after all.

Plausibly it could lead to weapons that kill many people but then many people have had such knowledge for a long time and we are all here.

Turns out mass damage requires things beyond just ‘intelligence’.

If you are suggesting there is some new model of physics or groundbreaking technology that would allow such a dragless plane, then don’t let me discourage you! But AI won’t help at all, since it will only regurgitate conventional wisdom…

Well yeah... how would there be fewer??

But the point itself is silly. Few people are putting effort into Maths for the fun of it (and of those that are many derive fun from being the only one who can produce a solution). Chess differs in that it never had any point but the game its self.

If my friend showed me some unpublished work (Say 90% of the hardest work toward a significant proof) if I then go finish the remaining 10% and publish the whole thing as mine it isn’t ‘building on the shoulders of giants’ it’s plagiarism/theft.

If he’d published that work first and I took it and found another proof and credited him for his work via citation then that would be fine.

It’s more like you spend 4 years developing a product you’re passionate about. This product will gain you the respect of all your colleagues and either earn you money directly or lead to great career advancements. Then OpenAI takes it, changes the colour scheme, finishes the login flow and claims the whole thing as their own.

Not only would it piss you off but it would also misrepresent what OpenAIs models are capable of.

And it’s all irrelevant. If one human on earth can consistently get it right then it hasn’t been passed since clearly that human can somehow determine between them (whereas no one would ever be able to determine between a true “human intelligence” by definition).

And as it stands almost everyone could tell between them when allowed to discuss whatever they want for any length of time.

The fact these researchers have to keep adding bounds shows it hasn’t been passed. If we are arguing over technicalities maybe it isn’t as obviously intelligent as claimed!

And people forget that sometimes humans message twice. An LLM can only respond. So it immediately fails here in a true Turing test. (You could loop the LLM but then I expect even more immediately obvious bot behaviour).

So the first thing they’d do is tell me LLMs exist and the other thing is an LLM. Obviously a true human level ai could explain that away as a fabrication to trick me. I don’t think an LLM could do even this!

Turings whole point was that through the medium of text along if the human and machine were indistinguishable then that was true intelligence. So yes conversations of arbitrary length are allowed (needed).

If someone sat me down today with an LLM and a human and both were trying to prove to me they were human, and I can have conversations of arbitrary length, I’d get it right every time.

So there is more to lose than just the service being unreliable.

Having said that, for all our startups “interesting” code I use no AI, not even tab complete. For a few reasons:

- having derived the code and built a mental model is something like 90% of the work, the code artefact being 10%. (You can probably verify this yourself, if you’ve implemented something once and then someone deleted it all, you could rewrite it MUCH faster the second time).

- once I have this model / vision in my mind, I’m streaming it from my mind into reality via code and having code suggestions pop up breaks that flow state for me.

- using Claude code even on throwaway scripts can mask problems. Why is doing this thing so difficult that it needs an AI to write the code? And usually when I’m writing those scripts I have a thought or insight about the system. in other words it’s nice to have that low stakes time.

- models are still frustratingly bad at spatial reasoning and I don’t see this improving anytime soon, which is a problem for a lot of graphics code.

- the code models produce still isn’t what I’d consider great code; subjective of course but it’s our product and we think having great code in our core technology is well worth it in the long term!

I do still use Claude code on various commoditised pieces (account UI, etc) and I use Claude as a great search engine, though its explanations of complex topics still kind of sucks and a human blog is almost always better.

We are a small company of 15 engs.

Why? If you genuinely believe these things righteous and important why can’t everyone do them?

Perhaps because you’re incentivised not to: it’s a waste of your time, someone else can do your share of the campaigning, non-vegan food tastes great etc..

This is exactly how incentives work, not some spineless person in a dark room saying “yes m’lord I’ll torture another child for an extra nickel” as the article portrays.