The screenshots in the Steam page do look impressive though! As an outsider, I could mistake this for a recent Civ game. Congrats on making it here!

For extracting the fractal from the residual stream, did I understand it correctly as follows: You repeatedly sample the transformer, each time recording the actual internal state of the HMM and the (higher-dimensional) residual stream. Then you perform a linear regression to obtain a projection matrix from residual stream vector to HMM state vector.

If so, then doesn't that risk "finding" something that isn't necessarily there? While I think/agree that the structure of the mixed state representation is obviously represented in the transformer in this case, in general I don't think that, strictly speaking, finding a particular kind of structure when projecting transformer "state" into known world "state" is proof that the transformer models the world states and its beliefs about the world states in that same way. Think "correlation is not causation". Maybe this is splitting hairs (because, in effect, what does it matter how exactly the transformer "works" when we can "see" the expected mixed state structure inside it), but I am slightly concerned that we introduce our knowledge of the world through the linear regression.

Like, consider a world with two indistinguishable states (among others), and a predictor that (noisily) models those two with just one equivalent state. Wouldn't the linear regression/projection of predictor states into world states risk "discovering" the two world states in the predictor, which don't actually exist there in isolation at all?

Again, I'm not doubting the conclusions/explanation of how, in the article, that transformer models that world. I am only hypothesizing that, for more complex examples with more "messy" worlds, looking for the best projection into the known world states is dangerous: It presupposes that the world states form a true subspace of the residual stream states (or equivalent).

Would be happy to be convinced that there is something deeper that I'm missing here. :)

Comma operator I would agree is really only used for shenanigans. But operator= is as crucial as the copy/move constructors for ownership, so I'm not sure how you picture this...

> apparently in C++ a structure cannot really have zero length

Yes it can: https://en.cppreference.com/w/cpp/language/attributes/no_uni...

> I think it should be possible for your own structure or union type to overload any operators that are not already defined

I see that you are trying to solve the problem of "when reading this code, I don't know if these are the built-in operators or custom ones". I would be interested in learning about situations in which this was a problem for you, because I have not made that experience myself. From my perspective, you need to know the types of x and y to know in the first place which operations are possible - default or custom - so this rule does not really buy you much. But again, I don't think I understand your problem well enough.

FWIW, you can define a custom operator-> and operator* (they have to return pointers, or types that themselves have operator->/operator* which are then applied recursively). I am not convinced that mixing pointer-related semantics into e.g. object assignment is a sufficiently elegant solution from a conceptual perspective though.

Sure, I agree, you should carefully evaluate which language best serves your needs, and what tools in a language you should use or stay away from. But some game dev ranting about which C++ features they consider useless (without even articulating their background or constraints) is worthless imo. Especially if this is the credit they give to their name:

> After over 20 years working with C/C++ I finally got clear idea how header files need to be organized.

[1] Yes, that FAQ parody is now very outdated, seeing as it precedes even C++11. I can believe that it was on-point at the time of writing, but C++ grew up since then.

Is there anyone who is seriously contesting the "it is intended to cure/prevent a disease, therefore it is a drug, therefore it needs FDA approval to be sold legally" line of reasoning?

More humoristically: https://xkcd.com/2475/ and https://xkcd.com/2530/

Less humoristically: https://en.wikipedia.org/wiki/Thalidomide

However, I don't know why you are comparing a single billionaire vs a single X kid household. Like, the number of each (or even of private jets) are not even _remotely_ in the same ballpark. Which is why "number of kids" is not at all a strange place to focus on environmental impact, but "billionaire lifestyle choices" is.

Comments URL: https://news.ycombinator.com/item?id=39629520

Points: 2

# Comments: 0

Comments URL: https://news.ycombinator.com/item?id=39503765

Points: 3

# Comments: 0

I would like to also highlight this article by Joel Spolsky: https://www.joelonsoftware.com/2006/04/11/the-development-ab... (It's concerned with the everything else in a company, but it applies equally to the senior vs junior responsibilities imo).

If you are under the impression that "writing good code" is the be-all-end-all of being a software engineer, and that being a senior means you get to enjoy it more, then you're probably working in a company that is really good at maintaining this part of the abstraction layer.

That's because the more senior you get, the more you inevitably get to be part of the system that allows more junior devs to keep their mind on writing good code. That means dealing with the meetings, timelines, uncertainties, bug triage, maintenance, documentation, compliance and customer/expectation management. So they don't have to.

Simpler example: Dreams.

(The other models being only partially able to source good references is unsurprising/"unfair" on a technical level, but that's not relevant for assessing their safety.)

"We assume 64 bit overflow is not going to happen because nobody can store that many bytes" could be valid if the existence of those bytes was required for reaching this code. But if user input can lead to UB being triggered here, fixing the code is indeed prudent, even if everyone were fully convinced that current compilers are not outsmarting themselves.