Which makes it absolutely unbearable. By the way, that combination of temperature + humidity will cause severe hyperthermia (which can be deadly) faster than people think.

Basically, the sauna studies are probably mostly discovering that "healthier people can stand sauna longer". In countries where most people don't stand sauna for more than a few minutes, that self-selection bias won't exist.

That sentence is doing a lot of heavy lifting.

- What's "disproportionate to its history"? Obviously something going from 1mm to 10cm is worth checking out, but what about something going from 1mm to 2mm? Might be a tumor, might be that the position is just slightly different.

- What about other less measurable factors? Example, border features. That's harder to measure and things like movement or different machines can change how the borders of a feature look. How do you know what's a baseline and what's not.

- How frequently do you run these scans? It's likely that if something "starts changing in size" suddenly it will start giving symptoms before you have your next scheduled scan.

> It's certainly not "throw more data at the problem". Instead it's about giving the data a time axis with some decent fidelity.

It's definitely throwing more data at the problem, and you're assuming that it's viable to give "a time axis with decent fidelity". MRIs are much more complicated to interpret than people think, and screening is a much harder problem too. There are a lot of studies testing MRI imaging as a screening technique (among other techniques) and they don't always show an increase in survival rates.

Not always. There are bunch of studies for MRI screening in high-risk populations for specific cancers. There are scoring systems for a lot of them based on imaging features and they do find asymptomatic cancers.

In fact, if you add low-risk populations to the studies used to design imaging scores, you might end up adding more noise and making the study more difficult and the scoring less accurate.

How? How do you establish baselines? How do you build a classification of incidental findings? It's very possible that you'll find a lot of types and not a lot of representatives of each type. And then you have to correlate that to actual clinical results, but the population will be so heterogeneous that it'll be really hard to find an actual result.

It's not just "let's throw more data at the problem".

What you're failing to capture is that this is a hard problem because it's both an individual choice and a collective one as well. Those "terrible side effects" might actually end up killing someone. You're choosing between a high-chance lottery on a small population or a low chance lottery on a far larger one. It's not that simple.

Doctors have already thought of this. Several issues with it:

* Monitoring still causes anxiety and mental health issues which come with real effects on patient's quality of life. It's not "harmless".

* Unclear when to monitor and when to treat. It's also really hard to get enough data to characterize these early unspecific findings enough to get confidence on what to do.

* Monitoring via MRI might be just as useful as monitoring via symptoms or any other "passive" methods that do not require a previous scan.

This is not what happens. You're assuming that if the cancer does not get detected by the screening then it never gets detected. What actually happens is that the test gives information that might actually be redundant and obtainable in less risky way. What the studies are showing is that waiting until there are other, more specific signs and symptoms of the prostate cancer results in the same survival rates.

How do you know which ones to treat and which ones to leave?

Why? What happens if the cancer still doesn't respond to treatment even when detected early? Or, to the contrary, if the cancer also responds to treatment when it starts becoming symptomatic?

That's why we have studies to understand if screening is a good practice or not. It's not that clear cut.

Why is it likely? We already have a lot of MRI data. There are already a lot of incidental findings. It might also be an issue of the MRI not being able to produce enough information to discriminate.

> To use a software analogy, if your downtime detection system kept producing false negatives, would your solution to be just turn it off? You'd get some better night's sleep, but you'd pay for it when the system really went down and you had no idea.

The analogy is rather something like this: your downtime detector is not just a "ping" but a full web browser that tests everything and it sometimes flags things that are not actually issues. So you don't turn it off, but you only use it when you have another signal that indicates that something might be going wrong.

* The cancer is aggressive and resistant to treatment. Chemo/radiation only pause the growth for a bit, but ultimately the cancer keeps growing and the total survival time is the same (only that the patient spent more time knowing they had cancer).

* The cancer is susceptible enough to treatment that it's still curable when it becomes symptomatic and found through other means.

* The cancer is slow enough that the patient dies from other causes before.

Early screening brings benefits only when the cancer ends up causing issues and responds differently to treatment between the "early screening detection" time and the "normal detection" time.

It's impossible to know beforehand which of the scenarios have more weight, specially because we have very little data on what happens way before cancer is detected via the usual methods. We need better studies on this, and for now the evidence doesn't really point out to these large, indiscriminate screenings being actually helpful.

- AIs should not take issues that are designed to onboard first time contributors - Experienced matplotlib mantainers should not take issues that are designed to onboard first time contributors

Sounds about the same

Of course, but we're trying to assess the capabilities by looking at the LLM output as if it were a program written by a person. If someone told me to check out their new C compiler that can build the kernel, I'd assume that other basic things, such as not compiling incorrect programs, are already pretty much covered. But with an LLM we can't assume that. We need to really check what's happening and not trust the agent's word for it.

And the reason why it's important it's because we really need to check whether it's actually "better than it was" or just "doing things incorrectly for longer". Let's say your goal was writing a gcc replacement. Does this autonomous pipeline get you closer? Or does it just get you farther away through the wrong path? Considering that it's full of bugs and incomplete implementations and cannot be changed without things breaking down, I'd say it seems to be the latter.

I mean, $20k in tokens, plus the supervision by the author to keep things running, plus the number of people that got involved according to the article... doesn't look like "a weekend project".

> Building a C compiler which can correctly compile (maybe not link) the modern linux kernel is damn hard.

Is it correctly compiling it? Several people have pointed out that the compiler will not emit errors for clearly invalid code. What code is it actually generating?

> Building a C compiler which can correctly compile sqlite and pass the test suite at any speed is damn hard.

It's even harder to have a C compiler that can correctly compile SQLite and pass the test suite but then the SQLite binary itself fails to execute certain queries (see https://github.com/anthropics/claudes-c-compiler/issues/74).

> which, in comparison with a correct modern C compiler, is far less performance critical, complex, broad, etc.

That code might be less complex for us, but more complex for an LLM if it has to deal with lots of domain-specific context and without a test suite that has been developed for 40 years.

Also, if the end result of the LLM has the same problem that Anthropic concedes here, which is that the project is so fragile that bug fixes or improvements are really hard/almost impossible, that still matters.

> it really seems that the complaints here aren't about the LLMs themselves, or the agents, but about what people/organizations do with them, which is then a complaint about people, but not the technology

It's a discussion about what the LLMs can actually do and how people represent those achievements. We're point out that LLMs, without human supervision, generate bad code, code that's hard to change, with modifications specifically made to address failing tests without challenging the underlying assumptions, code that's inconsistent and hard to understand even for the LLMs.

But some people are taking whatever the LLM outputs at face value, and then claiming some capabilities of the models that are not really there. They're still not viable for using without human supervision, and because the AI labs are focusing on synthetic benchmarks, they're creating models that are better at pushing through crappy code to achieve a goal.