Personally I’m betting that we will see a major comeback in local radio and print media, as global network effects regress local with collective trust.

The interest in a shared problem and the passion to solve it?

> A patent is the best thing we’ve collectively come up with

I think that’s a bold claim in the face of the numerous contributions from e.g. universities, open source communities, single-person projects, etc that have pushed the world forward. We simply do not know a world free of the oppression of patents, the institution sustains itself.

> If you don’t like the game, get better at it.

I don’t think people will be playing this game much longer in the face of generative AI, unless ofc the ruling classes succeed in locking down access to that tech, gobally.

Would love to know how they plan to do this, sounds like a patch that is bypassed with a $0.01 resistor.

Markets defined entirely by law have distorted our collective understanding of what can actually be built with the knowledge our species has accumulated thus far. How will traditional shields that have protected capital accumulation in tech to survive in a world where governments now realize control of technology is a national priority? Especially as we see its impact on modern warfare, and that such conflict looks like it’s only escalating over time.

Mathematicians appear to me (as an outsider) to exist in a field without such distortions, and I think offer engineers a preview of what’s to come. I certainly have completely ceased sharing original ideas online at this point.

It is broader than that though, and is the distinction I sought to capture with “anyone with access” in my original post. They are connected to the internet, and to a cloud no less! Anyone could be the NSA now, from anywhere in the world. I bet we’ll see these used to collect marketing data too.

Perhaps a limiting principle can be that we simply do not use ALPRs or other surveillance tech, or at least that we don’t connect them to the internet and require a warrant to access and use their data? Personally, I’d just prefer cops enforce traffic law the old fashioned way.

As others have noted, I believe the brightline is less “general purpose computer…” and more “…that is remotely accessible & modifiable effectively anywhere in the world”. I worry not just about OEMs and cops, but also criminals, foreign intelligence agencies, etc. Honestly potentially anyone at this point, now that folks have figured out you can use LLMs to do analysis.

This is a sound foundation for concern by itself, don’t you think?

The Super Bowl commercial that Ring put out about finding lost dogs spooked people for a reason!

You know, I haven’t kept up with MoE and etc where there’s a bit of selection going on, so I should probably be more humble. I think new work has only added different “paths down the same hill” though (no recurrence, just select different matrices), but could be wrong there. I don’t think I’m wrong on my general intuition, just want to be epistemically honest!

Each token prediction is one big function call. Then you just recursively generate more tokens until run out of context or the model predicts a next token indicating end of sequence. Technically the model outputs a matrix where the last row is a probability distribution, but I’m counting sampling from it as part of the chain. Hundreds of billions of dollars has gone into just making the function fatter and gradually changing pieces here and there.