I'd prefer the models to get better at SVG. I really hate working with the rasters that diffusion models generate, but the vector outputs are just really bad even when tokenizable like SVG. I've done some experimentation with trying to make these work better with some newer techniques with some success. But I also think the SVG Paths mini-language may be a bit too concise and unforgiving for LLMs to consistently get them right without specialized training.

Good Lord, what games have you been going to? I see a couple MLB games live every year and a few minor league games. I haven't seen anything like what you're describing. That doesn't mean there are no drunks or obnoxious fans, but that's the risk of doing something in public. The last game I saw live was San Francisco at Milwaukee in June. Giants won 1-0, and the home team crowd was fine. And I mean the Brewers fans like their beer, but in no way was it a zombie apocalypse. Last game I saw in Oracle Park last year was also fine. Even Boston a few years ago, which definitely has more of a college town vibe than SF or Milwaukee, was hardly zombie land.

I'll also say that AAA ball is typically awesome. The games are much cheaper than the big leagues, you get to see some big league talent rehabbing from time to time and see the occasional prospect who gets called up. They also tend to be very family friendly.

> Baseball has been a game of moral fiber, respectable players, and good clean fun. But it seems that activists and jerks set out to ruin the experience for people like me. I'm not having it. I'd have way more fun at a punk rock women's roller derby bout.

I don't know about that. Baseball has a certain level of honor, sometimes an excessive amount e.g. some of the "code" that has started to fade away the last decade or so. It also invented the concept of a nearly all-powerful commissioner, which was created specifically because players threw a World Series for a payout. It's a wonderful game, a thinking person's game, but it's always been quintessentially American for both good and ill ever since it scaled nationwide during the Civil War.

> > see it actually improve just through accreting context

> this actually happens and has been tested.

I specifically said a novel task outside of the explicit training. And I already agreed that the so-called thinking models do some level of logical reasoning. But being able to engage in some level of reasoning because it has learned logical inference rules doesn't mean it's actually thinking, regardless of what the researchers wish to call it.

Also, why does each model always fail at the two tests I give it? The models not only fail to improve, but they start to degrade after many subsequent iterations. Someone who can think would at least not get worse.

LLMs are filters or tuners for extremely subtle patterns, patterns that humans frankly are not great at finding. That's what the attention mechanism does: attend to the other tokens that are most related in a given context, even if that related context is distant in the token stream. Some patterns they fail to detect because they haven't been sufficiently trained or post-trained, and so the LLM just attends to noise (or at least that's what appears to be happening).

A lot of intelligence can be effectively mimicked through this pattern synthesis by transformer architecture alone. That's surprising. But I have yet to see them think.

The opposite happens in practice. I test new models with two tasks: iteratively generating SVGs based on a text description with rendered rasters for feedback; and generating "Before and After" clues like on Jeopardy, where the response has two overlapping phrases such that the last word of the first phrase must be identical to the first word of the last phrase. I have yet to find a model that is consistently good at either. And actually they tend to exhibit context rot with these tasks, where they seem drunk or stoned and the quality degrades.

They're extremely good pattern filters, and that includes some level of logical reasoning. But they aren't reflective or adaptable. Just last night, for instance, I was teaching my son about rounding to the nearest millions. It became clear that he didn't know the place values of large numbers, so we reviewed that till he was consistently correct, and then he was consistently great at rounding to the nearest millions or ten millions or hundred billions or whatever. He's thinking. LLMs are not.

I liked StackOverflow for the first ten years or so of its existence, but I gradually stopped using it then suddenly quit altogether when valid questions were being closed unreasonably. At this point, LLMs with documentation in the context, issue trackers and eve the source code (if available) have surpassed SO. Now my main issue is telling the LLM to crap on my idea rather than wishing it were kinder.

I've suspected that the energy regulations and the ruling party's close connection with Dominion Energy (the Governor recently attempted to fire the chair of Virginia Tech's board and replace him with the CEO of Dominion) have had an impact on power use more than data centers themselves.

Could your router handle this?

The design of the model is literally to find patterns and attend to them. The infrastructure and process around an OpenAI model is intended to filter "bad" things (in this case, I agree that the outputs are bad), but is designed to stop some enumerated-ish list of things that aren't allowed, perhaps with some limited "reasoning" about them.

The transformer was designed to attend to relevant pieces of context and generate new ones that match the pattern. OpenAI in particular was doing that work without guardrails, then attempted to bolt on "content filters," which in my opinion just can't work in a rigorous way. (I think Anthropic's "constitutional" approach is much better though not flawless. And regardless, Claude models don't generate images.)

So, yeah, working as designed. Maybe not as intended, because these things are somewhat resistant to the host's intent when the prompter is hostile.

But that's not what happened. The missing image was described as "graphic" or "violent." If I were to receive an email with that request and a missing attachment, my imagination certainly would not conjure images of butterflies & unicorns. Seems the model is working as designed.

I object. The CCP is much more deeply indebted than the US when taking into account provincial and local governments as well as state-owned enterprises.[0] And of course the US debt is financed in its own currency while Chinese foreign debt is financed in dollars or other currencies.

The problem in the US is regulation. An environmental impact study takes 54 months in the US.[1] The CCP, which has no problem poisoning its people or even launching rockets over inhabited villages, doesn't delay itself at all.[2] I'm glad we don't poison our people or place dangerous industry in places that could harm populated areas, or even perform some prophylactic measures to protect nature, but I'm confident that we could do this in less then a year (less than six months?) and make much faster progress. Even for something like nuclear, the ten years (mostly caused by red tape) are really onerous.

> China is the only one that can run if it comes down to it (unless of course the numbers coming out of China are mega bogus, but for that I don't know enough to have an opinion).

Yes, the common opinion among China watchers is that any number the CCP touches is "mega bogus." They're actually in the midst of something of a financial crisis at the moment because of the high debt.

[0]https://www.statista.com/topics/11662/debt-in-china/

[1]https://www.rff.org/publications/reports/how-long-does-it-ta...

[2]https://arstechnica.com/science/2019/11/china-keeps-dropping...

As far as I can tell, architecture, i.e. sound, precise definitions of exactly what a software artifact must do, is now critical. And with LLMs, it's now feasible to begin implementing such things, though many brownfield projects may be intrinsically unsound in ways that their creators are unaware of. In such a world, contributions simply require a modified proof that the software does what it must do, with perhaps additional claims that the maintainers provide.

[0]https://en.wikipedia.org/wiki/Software_crisis