The "random factoids" were verbatim training data though, one of their extractions was >1,000 tokens in length.

> GPT4 never merely regurgitated

I interpreted the claim that it can't "regurgitate training data" to mean that it can't reproduce verbatim a non-trivial amount of its training data. Based on how I've heard the word "regurgitate" used, if I were to rattle off the first page of some book from memory on request I think it would be fair to say I regurgitated it. I'm not trying to diminish how GPT does what it does, and I find what it does to be quite impressive.

While I very much do not think this is all they do, I don't think this statement is correct. Some research indicates that it is not:

https://not-just-memorization.github.io/extracting-training-...

Anecdotally, there were also a few examples I tried earlier this year (on GPT3.5 and GPT4) of being able to directly prompt for training data. They were patched out pretty quick but did work for a while. For example, asking for "fast inverse square root" without specifying anything else would give you the famous Quake III code character for character, including comments.

That GPT is so bad at tic-tac-toe and relatively good at other games like chess is one of the main things that contributes to me having a lower opinion of its ability to generalize than I would have otherwise.

I think any human with GPT's abilities in chess (but somehow no prior knowledge of ttt) would have zero issue becoming an expert with a single explanation of the game. Even very young children can learn to play ttt well and at least consistently make valid moves if nothing else.

Very impressive with almost everything else I gave it though.

Wouldn't say this is super reliable, I gave it a photo of a small squid in my hand and it said it was a baby fish (very obviously was not a fish).

I can't say I agree, this feels like a very uncharitable reading of his/her posts. Unless it was edited in after the fact they even said "You're absolutely free to believe either way, and I don't want anyone to force you to do anything."

While it would be insulting to call any individual person's preferences a result of brainwashing, I don't think it is a stretch to say that at a societal level preferences are shaped by mass-media and advertising. Improving access to and making people aware of less resource-intensive forms of comfort doesn't have to come from an authoritarian place. One of my major motivations for seeking out a more walkable area was urbanist YouTubers extolling the benefits. I suppose one could argue that things like bike lanes are hurting drivers but if a city's transit priorities stem from local politics and preferences I don't think it can be reasonably argued that making any particular transit method a priority is more authoritarian than another.

> In practice, comfort is mostly a function of stuff.

No question that it is a variable for most people but I'm not sure I buy that it is the most dominant one. All other variables being excluded, time to do what I want is at least as important for me as stuff (luckily I like my job so time/money aren't usually in conflict). And I think for many people "stuff" like cars and nice lawns aren't inherently drivers of comfort, but rather just possible reifications of goals like "pretty yard" or "fast/easy transit," both of which can be realized in less resource intensive ways. For the yard example, that might be a native garden or xeriscape (in some cases there are rules against these, which actually goes against freedom imo).

I moved from an area where I needed a car to an area where I don't and doing so increased my comfort. If areas like this were more accessible I think a lot of people would willingly degrowth and become more comfortable at the same time. Of course people shouldn't be forced to lose their car or move to a denser area if they don't want. And I like my gadgets but it is pretty ridiculous that their lifespans are artificially shortened to prop up profits. I have a computer from 1984 that still works, I would bet a huge amount of money none of the devices I buy today will work in 2062.

ex: "This code `(some-fn 1 2)` does x because y. That is incorrect because abc"

I wondered if this has to do with common StackOverflow post formats.

I was recently in El Salvador, which uses the USD as an official currency (alongside BTC lol). Despite using dollars, the $1 coins are used instead of notes almost exclusively for that denomination (mostly presidential dollars and some silver Susan B Anthony dollars). I was curious and did a bit of research, apparently the reason is that because day-to-day transactions are done almost entirely in small amounts of cash the paper notes have a very short lifespan (apparently <1yr) there, while coins will last decades.

The lead researcher behind the treatment claims the treatment cost under $20k to administer though, and I've never heard of mass production/adoption causing the cost of a product to go up in price (edit: excepting certain luxury goods that go up because they're a status symbol or something), certainly not >20x more... I know nothing about the medical field so I'm open to having my mind changed but that completely defies intuition. After seeing $450 bags of saline on a medical bill I had last year I'm much more inclined to believe they're just price gouging.

> Novartis is also not the only manufacturer providing CAR-T drugs at a high price.

I guess more than one company can do something despicable?

> offer alternative payment programmes

The "alternative payment programme" mentioned is "requiring payment only if the CAR T therapy induces a complete remission by a certain time point after treatment." I'd much rather pay $20k whether it succeeds or fails than $475k if it succeeds and $0 if it fails, if it fails I'm probably dead and don't care very much...

https://en.wikipedia.org/wiki/Population_history_of_the_Indi...