That's the alignment problem I'm referring to (which is one of the many aspects of alignment), for which we do not have robust recipes, and not only that but for which research suggests it is becoming harder to create guardrails for as base models get smarter.

I'm sure OpenAI would argue that base model + guardrail is aligned, but considering the "relative intelligence" of these two pieces, the fact that guardrails can just be turned off, and these kind of incidents, I am not reassured. We may well get another "oopsie" moment with much more catastrophic consequences even from otherwise well intentioned actors.

I think the symbolism of "let's crush all remaining vestiges of creative culture" is a pretty obvious _potential_ interpretation from a _non-trivial amount_ of people. In that sense it is an interpretation that matters for our present discourse, even if it isn't the interpretation that the creator of the ad intended.

Having been in both places (Brain and DM), this feels so far from what I experienced that I must ask, what are you basing this on?

https://www150.statcan.gc.ca/n1/pub/85-002-x/2021001/article... https://spvm.qc.ca/upload/02/2021_Activity_Report_SPVM_EN_VF...

Also please be more skeptical of thoughts of the form "X is Y because of hot take Z". Reality is complex and things have many causes. Montreal has a long history of doing city planning differently than other cities in Canada.

This is definitely the case in DL (and I'm assuming elsewhere too but I wouldn't know).

I've lost count honestly, running 1-2 year old paper github repos with some detail missing (like the Python version!) that make it non-trivial to run as is. Libraries make undocumented breaking changes, wrong pickle format, authors used a nightly version which didn't make it to a tagged version, and so on.

This perhaps says also something about the CS (versus software eng) background that most people engaging in DL publishing have.

Concretely what this could mean is using these tools to generate causal hypotheses, like what's been done here: https://arxiv.org/abs/2202.13903

- RL says, give me a reward and I'll give you its max.

- GFlowNet says, give me a reward and I'll give you all its modes (via p(x) \propto R(x)).

Yes you would ideally have a loss (well, a reward/energy) that is invariant and operates e.g. directly on the molecule rather than on some arbitrary ordering of the nodes.

But, yes in the moment it felt like some very serendipitous insight!

It may be possible to infer/learn a score from existing proofs though. We have a paper that manages to both learn a flow and an energy function (the score) from data: https://arxiv.org/abs/2202.01361

I don't know much about theorem proving though. Can some value be attributed to partial proofs?

Source: am first author of original GFlowNet paper.

Grad students are one of the very few subpopulations of humanity allowed to take on extraordinary epistemological risks; a kind of immune system of our civilization. I'm not even talking about some elusive notion of "progress", just [intellectual] societal health. It would feel to me like a tremendous loss if we let go of such a component of society.

Fighting climate change is not a game of "what would be more effective" and ranking solutions (especially in between countries), it's a game of "what are ALL the things we can realistically do". Both _must_ be done.