Normal welding is intentional application of heat to partially melt two parts at the seam, so that they "mix" in semi-liquid state and become one part when they solidify. Welding may or may not use a third material (solder) to aid the process.

For starters, we still need the AI (LLMs for now) to be more efficient, i.e. not require a datacenter to train and deploy. Yes, I know there are tiny models you can run on your home pc, but that's comparing a bycicle to a jet.

Second, for an AGI it meaningfully improve itself, it has to be smarter than not just any one person, but the sum total of all people it took to invent it. Until then no single AI can replace our human tech sphere of activity.

As long as there are limits to how smart an AI can get, there are places where humans can contribute economically. If there is ever to be a singularity, it's going to be a slow one, and large human AI vompanies will be part of the process for many decades still.

I wonder if there is some sort of transition between recalling declarative facts (some of which have been shown to be decodable from activations) on one hand and completing the sentence with the most fitting word on the other hand. The dream that "hallucination" can be eliminated requires that the two states be separable, yet it is not evident to me that these "facts" are at all accessible without a sentence to complete.

If you have one normal sentence and one overly verbose, the latter will have more tokens and therefore more weight.

It's missing a tree structure, so there is no ordering of skills (learn easy stuff before hard stuff because there is a learning curve to anything).

They might be good as prints to hang on your wall, but in the current state they're more "achievement lists" rather than anything resembling a tech tree.