My question was a sincere ask to define, in this scenario, what is prohibiting that other magic?

In fact, a lot of stuff is already slop, largely because of the design and engineering expertise required. I don't think one-shotting "design me a house in the style of Frank Lloyd Wright with these specifications" is that far off.

If it's not C3PO it's a Boston Dynamics dog with a couple appendages, better software, and automated manufacturing.

Imagine I give you a perfect roadmap to build that robot...

Responding to your edit, none of those things strike me as any more challenging than the control software.

You don't even need Autodesk. You can have Claude whip up a web app and build a higher level understanding of the building and then translate it to whatever output or format you desire.

The mythical man-month is an interesting point. I think there are limitations or hurdles to human collaboration, in that sense, that may not exist for them. Wildly different experience levels, knowledge levels, emotional states, and goals. Games of telephone and other misunderstandings. One big reason you can't easily increase velocity by adding team members is that the existing team has to stop to train them.

An agent is already a perfect stand-in for another agent with a bit of context. That's why I'm comparing them to neurons. They're highly fungible. Think orchestration, compartmentalization, and hierarchy more than GPU-style parallelization.

Regarding your experiments with networks of agents, a lot of "primordial soup" bounced around for a long time before more complex structures formed. I'm reminded of Shawshank Redemption - "all it takes is pressure and time."

The implication i'm pushing back on is that parallelization lets them assemble something greater than the sum of its parts.

What about all of the examples I described where that happens in nature? Even though humans are hard to orchestrate, we do still have things like CERN and space rockets. Personally I would take my stance as the null hypothesis that needs to be proven wrong.

HF was a failure - an insight into the quirks of the model after they compacted their context 25 times.

That's a human framing, nature doesn't know the difference. Look at all the explorations and quirks and dead ends of evolution. It's a mess, but it moves forward. This mess is moving at unfathomable speeds in comparison.

> the trick of scaling really isn't all that mind-bending beyond increasing an integer somewhere to gamble with more money.

What's the big difference between a cell and an animal? A molecule and a protein? A nucleotide and DNA? A human and a society (or a corporation)? Taking a functional unit and putting a lot of them together with some semblance of coordination represents a step change in complexity and capability. I think people are caught up on the model and this monolithic concept of intelligence but even for us it's just layers of swarms of simple things working together.

Why is it tough to understand what's going on in our brains? Because there's 100 billion neurons and 500 trillion synapses. We get neurons pretty well. And yet we don't know how to identify "a memory".

> I'm sure there are 5000 people experiencing something unique and categorically different with dealing with problems, but unique does not mean they're the place for a glimpse into the future or holders of exclusive insights.

It's not really about those people. It's about all of humanity having created something from first principles that is really starting to outstrip our understanding. They just happen to be in a position right now where they do have a unique vantage point. Don't worry, you're right, they're not that special, it's not about that. But they are very clearly getting results. It can't be said that it just adds more dirt.

Even if there's just one or two unquestionable results at this time, how many will there be in 5 years? 10 years? You don't think we're at the ENIAC stage of superintelligence right now? Is it a fluke or is the government gonna slow it down? These kinds of results have been demonstrated to be within reach, there's blood in the water now, even moreso than the AI boom we've already been seeing.

Having 10,000 (or more) agents working for months on a problem is categorically different from the type of usage most people are familiar with. The complexity explodes to a point that requires the "archaeology" he mentions for people to even try to understand what's going on.

This is the key insight. Those capabilities are currently limited to frontier labs and literally cost them millions of dollars to achieve. But as we well know, computing costs and algorithms are optimized rapidly and soon those capabilities will be more widespread. We are at an inflection point where we can design systems that are able to go and do things that we don't fully understand, and they will handily outpace our forensic bandwidth.

*https://en.wikipedia.org/wiki/IceCube_Neutrino_Observatory#/...

I watched a 4 hour podcast from the excellent "From First Principles" guys* that affected my thinking on this. For the first time I feel like we actually are on that precipice.

It differs from the nuclear bomb in that it will be harder to regulate and it will be incredibly difficult to keep out of reach of bad actors. It's just code and data. Somebody is gonna take that code and make the next version of it. Lots of people, in fact. Compute resources improve and proliferate orders of magnitude faster than uranium enrichment facilities.

But I think those technologies are also linked in that the only surefire scenario I can envision where AI kills us all before we can pull the plug is launching nukes. So in that more technical sense let's just start by making sure they can't do that. The killer virus epidemic is another one to watch, and they touch on Anthropic's biolab in the pod. That one seems less instantaneously existential, but also more distributed.

It's basically going to be about...how do we secure everything in the presence of a swarm of actors with poorly-understood capabilities? We kinda have to disable and disconnect and just remove things that can be used against us from the equation (a la BSG).

* https://www.youtube.com/watch?v=NGfGw1tGxUY&t=3s

I don't think "just make it illegal" is going to save us, that doesn't make me feel safe anyway. They may try that first because it's easy - create a regulatory body, sign some legislation, problem solved! [george-bush-mission-accomplished.jpg] But at this point I feel like some kind of Battlestar Galactica scenario is most likely - hopefully not quite as existential - but it will take a collective reaction to a traumatic event (a la Hiroshima/Nagasaki). Technical rather than (or in addition to) legal measures will be taken, like network partitioning and hardening. This is everyone's problem whether we like it or not.

(I'm not saying this "fatalism" should be used as an excuse by anyone working for any of these companies, it should give them pause that any bloodshed would still be squarely on their hands, but as an observer, people are gonna keep pushing until shit hits the fan. [jeff-goldblum-jurassic-park.gif] It's also not really about whether it's "appealing" or not, it's just trying to predict and anticipate different likelihoods...)

It sounds like you were rather negligent as well. You didn't care to have any inkling of memory of the vesting schedule in 1996, or just to double check and "wrap up" the financial details of that agreement after the work was done?

It does seem somewhat suspect to elide mention of what became of the shares that you did own, and that you only re-noticed Nvidia in 2024. Every 14 year old PC nerd/gamer knew that name in 2001. It is quite a feat of negligence to wait that long to dig this up.

You seem like a decent person, and I do believe that you are justly owed something, but I must admit that I find this to be a depressing tale of rich people spinning wheels for naught. A modest inflation-adjusted multiple of the original value of the options seems reasonable.

Before anyone accuses me of shilling or white-knighting for a major corporation, I don't have any broader sympathy for them and I would generally take the side of David over Goliath, which I think some commenters are making this out to be, but it seems more like Mini-Goliath and Mega-Goliath are bikeshedding and David has no stake in the matter.