This is a reality for all software, there is always a possibility of exploit so you design your environments accordingly (e.g do not have single points of failure)

Doesn’t this make it less profitable to be an R&D company, since there is less market share available?

I have two issues with the implication of this statement:

1. Meaning is inherently subjective. Reality is just a canvas on which sentient beings create their own meaning.

2. There are many, many examples of where “playing with symbols for their own sake” have yielded deep insights. There’s actually some implicit structure (eg the structure of logic) that is intrinsic to the universe we live in.

Who said anything about saving people from unemployment?

> I noticed unreliability during high load

This is pretty ambiguous. You still need to provide more context, such as which failure modes you are trying to benchmark.

You can believe they believe they are telling the truth or that they have good intentions while still being skeptical of the effective trajectory.

This is a really interesting point. This is almost analogous to a metric in software engineering like lines of code which is a proxy for something we actually want to measure.

In a way, the SOTA model providers are cargo culting what appears to be what mathematics is.

To slightly deflect your question, what do you think will arise naturally?

I’d put my money on feudalism if SOTA model providers can stay far ahead of open weights models. However if that proves infeasible and everyone ends up having access to local AGI does that serve to level the playing field? What resources are still going to be scarce in the coming decades?

See its not about pacing the frontier, its about pacing the frontier without hurting any of their fundraising.

If your point is that capitalism fucks up the incentive structure and makes it all about maximizing productivity then I wholeheartedly agree with you.