I have a the cure for cancer. Simply kill the host. Does it work? Yes. Have you learned anything from it? No.

My understanding is they are? And literally everything in this world is based around incentives. If you say “well you can continue to work on understanding, but your kids are going to starve” that’s not nothing.

It seems like a lot of the issue here is that these problems aren’t interesting in and of themselves, but they lead down interesting roads. It defeats the purpose if you solve them without getting any real understanding.

It’s akin to saying you’ve solved “pancake flipping” problems with a waffle maker, or “travelling salesman” problems with a zoom meeting.

But it also doesn’t mean these things aren’t problems, they’re obviously huge problems.

He just accepts anything that Claude says as truth. He vibecoded over 60,000 lines of code in 3 weeks, but couldn’t get it to do what he want and made a project overrun for 3 extra months. When the pissed off stakeholders called a meeting to ask what was going on he didn’t show up and sent his junior engineer to answer questions and take the blame. Now thats leadership.

But I mean they’re not good enough to do the things I need them to. Not enough that any difference can be chalked up to “taste”.

Put it this way, if you hired a very good handyman to build you a cathedral, you wouldn’t stand looking at the smouldering wreckage of the construction site saying “oh well that’s just a matter of taste”.

One thing that I’m particularly frustrated with is the writing quality of LLMs. Like this is the thing that they should be able to do, but I would say almost everything they write has almost no signal.

Over a mid sized AI generated codebase that means I’m reading like 500 words to figure out what a module is even doing.

In my experience (scientific programming) AI is a giant multiplier for people with specialized knowledge.

But it’s also a giant devaluer for that same knowledge as people with no idea what they’re doing can clog the field with plausible bullshit.

It’s now the case that if someone tells me they’ve done something, and I look into it and find out it’s completely AI slop, then I will have spent more time on the project than the person who “made” it. The situation is completely untenable and only serves to drain time and resources from people with better things to do.

Being able to run Jupyter cells independently is a feature until it’s not.

I’d say for most of my one-off work, it’s fine. But for stuff I want to share it’s not.

I tried experimenting with what is ultimately a treesitter based approach - https://github.com/0x007BA7/codebook

And really liked it. Definitely nowhere near production ready but I think theres room for a player to come in and do something similar.

It’s not much of a moat, but it’s more than a lot of orgs have.

But I switched from ChatGPT to Claude 3 months ago because my account was down for like 6 hours. I haven’t used it since. It’s too easy to switch away from chatbots on a whim. There is no moat for that.

But maybe you should checkout the tools it’s based on, sem - https://github.com/Ataraxy-Labs/sem and ultimately treesitter. They at least give a more structured approach to dealing with code than simple text.

It’s a better code reader built on top of sem (treesitter). I’m getting a lot of massive PRs at work now, and this has helped a lot with reading them. It decomposes the changes into entities and sorts based on what has the most dependencies. This tends to put the most important functions first. Plus I can click through the dependencies for each function and mark things as reviewed as I’m reading them. It’s a big improvement over the GitHub review flow for me at least.

It’s always a little disappointing to me when I think I’ve run into something unique but it ends up being user error or something.

My metric is: how much code that I create today do I depend on in 3 months (incl learnings and one off data pulls etc.) and its waaay less than what I generate. A lot of it is frankly noise.

If you have particularly specific knowledge in pretty much any domain, combining that with AI can lead to huge gains.