And the problem is that as long as humans are in the loop building software that dynamic will have to be maintained.

We use Loki for logging at work. There’s certain types of queries it just doesn’t support. And so the question becomes -

Do you change logging providers Adapt to Loki’s capabilities Create a third layer / tiered storage.

And each of those decisions have multiple downstream consequences. It’s not that LLMs can’t make those decisions per say, it’s that

What does an LLM do when five different people ask for a system optimized to do five different things.

It could figure it out itself, but like I don’t think that’s how the human software contract works.

None of this requires expertise, prompting, or mention of TDD. It's the default.”

I’m pretty sure it takes some level of expertise just to use the term “instrumentation” correctly in a sentence.

“Introduced in 1918, Radithor was William J. A. Bailey's biggest commercial success, selling 400,000 bottles between 1925 and 1930. The Food and Drug Administration (FDA) issued warnings against the use of Radithor but did not have the authority to ban it during this period.”

Many of these examples seem less like settled science and more like a set of fads. But it also seems to leave out all of the fringe things people tried and didn’t work.

While I’m not completely obtuse I don’t actually understand the point this person is trying to make other than knowledge is hard and political.

“1930s: a chemist dissolves a new sulfa drug in diethylene glycol, which is antifreeze. It kills 107 people, mostly children, and only then does America give the FDA power to demand safety testing.”

Is this a pro-institutional screed or an anti-institutional one?

I feel like what’s weird is people don’t really have a solution here. We’re simultaneously criticizing cutting edge science while simultaneously trying to lower the bar around getting things like “peptides” to market.

Things are easier to validate in person so there's a sense you understand the context. If you go to a painting club because you want to learn how to paint better, and someone or someone(s) show up with photography instead. You don't hate photographers you're just there to do a specific thing. Or say a club that uses traditional woodworking tools. You may not hate power tools or the people who use them, but they're not invited to the club.

I wonder if all the noise is around a need for better tools. Some level of consternation is good to signal that there is a problem in need of a solution. Hopefully there are solutions to this that aren't too invasive, but I think the counter-argument that it's wrong to have a like preference for human generated content is like - I think it should be ok to want to try and specifically define what a community is for.

But the reason you don’t have to read assembly when you write in a higher language is that, over time that abstraction has become largely complete and is deterministic.

Like when you write I dunno Python to add two numbers and calls class, that spec is universal and agreed upon. ~99.99% of the time when you read Python documentation, and replicate it, it does what the designers intended.

Spec driven development is none of those things. So while I’d like to see people experiment with higher level languages, I think many folks find there’s now way to consistently reconcile the spec with the code.

As much as no one wants to read the code anymore, it seems like most people seem to agree it’s the only source of truth.

But I imagine there’s this problem that measuring intelligence in AI is not the same as measuring intelligence in humans. How is fable not already super intelligence. How is it not way smarter than the 99th percentile human? In terms of speed, broad ability and pattern matching behavior. If intelligence is loosely correlated with lifetime income, Anthropic billions in revenue should be a decent measure that LLMs are way smarter than most folks.

Because sometimes I think the inconvenient truth is that when people ask, you don’t think we’ll have AI that’s smarter than humans. It feels to me like we had AI that was smarter than humans a long time ago that somewhere around GPT three or GPT four we had AGI and if we already have AGI and we already have AI that’s smarter than humans then maybe the real question is is intelligence not his intelligence actually the right measurement for what we’re trying to achieve.

Or maybe it’s that ASI inevitability claims to be a very general argument, when it’s a very specific argument (or vice versa)

The AI inevitability argument seems to be something like - are you so anthropocentric and unimaginative that you can’t see a world where human creations outdo humans.

When the specific claim is more like “The specific technology of large language models will hard takeoff in the next few years”

There are infinite years ahead of us. Maybe hard takeoff is coming! And maybe it’s a thousand years away.

Cara are great! But a one hour drive serves a different purpose from a one hour run. Running doesn’t make me a Luddite.

So I think - it will continue to be important to “do the reps” so to speaks. Build mental models. Read long form. Do crossword puzzles. Think on your feet and have in person conversations.

I think / if a university carries the responsibility of credentialing you, they equally carry the responsibility of ensuring you didn’t bring a fork lift to the gym.

Individual ability? Newer engineers should read and write more code than experienced engineers.

Maturity model? Teams with better access to objective specifications can read less code than teams without it. Teams with more experience with building loops and validation frameworks the same.

Industry tooling? I think we need more work on the spec driven development front. Some set of standards as to how we define contracts that LLMs can operate against that are more deterministic.

In the context of many other measures I think speed is an important measure. Maybe even the key measure. I’ve even written a blog post entitled “reading code is an anti-pattern.”

But if the metric is, slowing us down from having more lines in production then the answer is unequivocally yes.

If the answer is providing customer value, or having a sustainable engineering culture. Then I’m not so sure. I’m not saying I believe the opposite, but it feels like excessively optimizing on the wrong thing.

Ultimately I think this asks the wrong question, I think most other surrounding questions is the right one which is - how do you safely and quickly deploy the right code to production that delivers customer value.

I think that will continue going forward involve doing so in an automated fashion, but then the right question isn’t, “should we stop reading code”. But something like “what is the right way to ship intent to production”

Because if you do the first without the second…

And, as someone who does a decent amount of vibe coding -

Are we really turning engineering into plain text and not a like agreeably defined spec? Is that how we’re writing applications now?

That a list of instructions that may not have an agreed upon definition or compilation process to ensure they’re internally consistent - that’s the foundation of modern change management?

It's not the usage of AI so much as a seeming enthusiasm that this generation of tools are necessarily the right ones.

It's kinda weird because I think this almost loops back on itself, but a fundamental question is like - what is the difference between AI and a library?

What's the difference between typing say "I want to reach out to the google API and get a json blob of my e-mail" and "axios.get(google.com/e-mail)"

What's the difference between "Build me a login screen" and having a good set of react components you can quickly compose into a login screen.

As sort of illustrated in the article. It seems more reliable and potentially less expensive to have a layer that can sort of say "This line here, it's covered by this library/git hook, we'll actually pull it out, replace it with a deterministic layer and maybe put it in a spec that's ignored or verified by LLMs to have the deterministic component, but not interpreted by LLMs"

Like going back to the early 90’s there was this incredible rash of bad films, especially action movies, and that was just the nature of Hollywood incentives.

Because I think the complaint in some sense isn’t the fact that there are progressive movies, it’s the fact that it’s a layer added to all low risk big budget studio films.

But these movies would likely be bad for other identifiable reasons if not for the sprinkling of diversity. The lord of the rings on Amazon isn’t otherwise great.

So sometimes I think this criticism is just feeding into the very thing it’s trying to fight. Industry is great, and it’s great for the reasons described, but I think the moral ambiguity problem as described isn’t caused by diversity, they’re correlated. Bad movies have less moral ambiguity and use diversity in a hamfisted way.

Scott says that one of the great things about superforcasters is that some people who regularly make bad decisions will be able to make better decisions by asking superforcasters questions like “Is this marriage likely to end in divorce.”

And there’s a number of profound questions in here about - what are we all here for?

There’s something of a philosophical horseshoe theory here which is like - I might argue - this sounds like astrology or other metaphysical forms of divination.

And the counter argument would be something like - no it’s completely different because this is true. Which is of course what all the spiritual traditions say about each other.

So it begs the question - is it the solution or the process.

Is using a super forecaster better because it’s “real”; or like I think I could make the argument it’s worse for that reason. Asking your grandmother or spiritual advisor is part of a process of existence. You ask multiple people, you have the experience, maybe you marry the wrong person, you learn and you grow.

But when you outsource that decision making.

But then if this happens - all of the stock market has risen in the promise of AI. If AI eats profits instead of grows them, then the economy shrinks right? So maybe that’s worse? That there is no productivity increase?

It's quite interesting to me that your epistemology is basically "God in a box is inevitable" and not skepticism.

> Now if you have millions of instances running in parallel, all "probabilistic", working on frontier AI research I really don't see the blocker (and believe me I wish I did).

The blocker is lots of things, but for one? Compute. Millions of instances running parallel, all "probabilistic" testing different theories would require million of GPU clusters.

Two? For subjects like Physics - Atoms. You need to be able to run experiments. You need access to the real world.

Then there's the S curve. Algorithmic breakthroughs operate on orders of magnitude. Eventually we'll run out.

etc. et al.

"This, then, is my prediction for the AI superforecaster future: for basic questions, your off-the-shelf AI chatbot will be able to offer opinionated probabilities superior to those of any human. For more controversial or bias-laden questions, a new era of prediction markets will smooth over differences in brand and model and efficiently aggregate all AIs’ opinions."

But one reason that you would publish it for everyone else to react was exactly what I said, because if you can show people that a superhuman AI believes a certain outcome is more likely, you impact whether or not that certain outcome is more likely. Which feels like a flaw in this version of the future. That in fact you could overcorrect and make the markets less efficient.

But I hate when guidance is totally verboten because like -

“Babies should not be shaken at all in the first two years of life”. I think has a _significantly_ different impact meaning then when you say it about screens. I don’t think if my baby happens to glance over at the TV there will be significant impacts to health and development.

And I know that, but when you say it’s just verboten, it doesn’t actually provide any signal, and it’s similarly used for things that are less easily differentiated.

As soon as the prediction market says “this path here has a 25% chance of curing cancer” all sorts of money is moved away from other things.

It will absolutely cause political outcomes not just predict them.

And then of course there’s the cheating element. Anything that’s feasible to change the outcome.

Maybe this just contributes to efficient markets? Or maybe the continued quests towards utopia have dystopic externalities.

But that is not a Pulumi problem.

That is a software design problem."

An uncomfortable idea is that these are basically the same thing. There's a reason the phrase "just enough rope to hang yourself" exists. A system that requires more software design a problem in and of itself.