That is not what the first graphs show - the Anthropic models cluster at 'better' positions on the graph, and I imagine you could show that the values are significantly different.

That's just in that final graph, and that graph is perhaps the least instructive - they talk about ranges of outcomes but they don't show whether all of the models besides Mythos / Opus 4.6 overlap

Take a look at all three graphs together and it's clear Anthropic are doing better in this arena

> On September 18, 2020, the Justice Department unsealed a seven-count indictment charging Garrison with “staging over fifty accidents.” Alfortish and Motta weren’t indicted or named in the document, but they were described, respectively, as “Co-Conspirator A” and “Attorney B.” Garrison’s coöperation with the F.B.I. wasn’t referenced in the text—and it might have seemed that charging him in such a public fashion would be a good way to conceal his role as an informant. But a close reading of the filing encouraged certain inferences. One stray sentence asserted that “Co-Conspirator A instructed Garrison on the number of passengers to include in staged collisions.” Alfortish might have made some unconventional life choices, but he wasn’t a total idiot. He certainly hadn’t supplied that information to the Feds—and the only other person who could have done so was Garrison.

> Four days after the indictment was made public, Garrison had dinner with his mother, Sandra Fontenette, who was seventy-four, at the tidy condominium that she owned, on Foy Street. They ate gumbo and talked. Garrison had been texting with a woman named Kim that afternoon, and they had made plans to hang out after dinner. At around eight-thirty, the doorbell rang, and Garrison went to meet her. But, upon opening the front door, he shouted to his mother, “Get down!” Ten shots rang out, and Garrison collapsed on the floor, dead.

Whether the difference is meaningful can’t be determined from the graphs (and picking one graph over the ensemble also doesn't have a reasoned basis given that these are all arbitrary).

I don't quite see why the author shuns them.

It's not your fault that you don't know this, but this whole subthread is very CS-coded in its disdain for other software people's standard of evidence.

Rent-seeking of old was a ground rent, monies paid for the land without considering the building that was on it.

Residential rents today often have implied warrants because of modern law, so your landlord is essentially selling you a service at a particular location.

Yes, it is far inferior to the 'Trust torginus and his ability to understand the large body of experience that other actual subject-matter-experts have somehow not understood' strategy

>No, it's not. It took years of polishing by software engineers, who understand this exact profession to get models where they are now

This reads as defensive. The thing that is easy to learn is 'why are biology ai LLMs dangerous chatgpt claude'. I have never googled this before, so I'll do this with the reader, live. I'm applying a date cutoff of 12/31/24 by the way.

Here, dear reader, are the first five links. I wish I were lying about this:

- https://sciencebusiness.net/news/ai/scientists-grapple-risk-...

- https://www.governance.ai/analysis/managing-risks-from-ai-en...

- https://gssr.georgetown.edu/the-forum/topics/biosec/the-doub...

- https://www.vox.com/future-perfect/23820331/chatgpt-bioterro...

- https://www.reddit.com/r/ClaudeAI/comments/1de8qkv/awareness...

I don't know about you, but that counts as easy to me.

-----

> I would apply this framework to biology - this time, expert effort, and millions of GPU hours and a giant corpus that is open source clearly has not been involved in biology.

I've been getting good programming and molecular biology results out of these back to GPT3.5.

I don't know what to tell you—if you really wanted to understand the importance, you'd know already.

It's very easy to learn more about this if it's seriously a question you have.

I don't quite follow why you think that you are so much more thoughtful than Anthropic/OpenAI/Google such that you agree that LLMs can't autonomously create very bad things but—in this area that is not your domain of expertise—you disagree and insist that LLMs cannot create damaging things autonomously in biology.

I will be charitable and reframe your question for you: is outputting a sequence of tokens, let's call them characters, by LLM dangerous? Clearly not, we have to figure out what interpreter is being used, download runtimes etc.

Is outputting a sequence of tokens, let's call them DNA bases, by LLM dangerous? What if we call them RNA bases? Amino acids? What if we're able to send our token output to a machine that automatically synthesizes the relevant molecules?

In all fairness, I've been accused of sounding like an LLM this year, which is quite unfortunate as I think we're coming to the end of careful writing.

https://lalitm.com/post/building-syntaqlite-ai/

Flags for LLM vs human drafting:

- Subtitles have the rhetoric turned to 11 with LLMs. (Note: Who has ever had multiple sentences as a blog post heading? It's bizarre) :

  - LLM "The Demo Works. Production Does Not."

  - Human "AI is why this project exist, and why it's as complete as it is"

  - LLM "Six months ago, a practitioner could name a preferred OCR engine with confidence. Based on what I read, that confidence is gone." - *What was read?*

  - Human "AI coding tools and playing slot machines"[ref]

  - LLM *Scroll through—each thing is about the same length*

There are lots of tells like this. This is a moment to get good at detecting LLM text in case it's surreptitiously used to your detriment.

When I use an LLM, it tries to sound like me but there are still tendencies it falls back on, especially when the context window begins to expand.

The 'missing subject nouns' is probably the LLM's way of sounding like an authoritative source in a technical field since many programmers like to write that way.

Never published an LLM text, friend.

And if somebody needs Claude to get something published, that person should find a better line of business, one more suited to her or his aptitudes.

> I would be genuinely interested in specific changes you would do if you were the editor.

This whole thing would get sent back with the kind request to think of an argument and write it out. By hand. Without an LLM.

How else do you think I would have come to write this comment? I got to the second major heading before realizing that there is little human input in this document.

I use LLMs but I will never impose on Claude's intellectual musings on another person as some sort of intellectual insight.

This is about the same as copying someone else's homework and then presenting the copied work as an example of deep brilliance. The copying isn't great, but the boasting is absurd. Who are we trying to con?

Maybe, but it felt like this was meant to support the idea that the company is scrappy / under construction.

Comments URL: https://news.ycombinator.com/item?id=47601055

Points: 7

# Comments: 0