The comparisons to nuclear and biological weapons seem to me pretty apt, and I don't think we should be so easy to discount the fears of people with a lot of inside knowledge and expertise on how things are progressing and where they might be headed.

Here ya go: - https://github.com/PhiloLabs/fable51-worlds/blob/main/union-... - https://github.com/PhiloLabs/fable51-worlds/blob/main/union-...

I really appreciated the negative framing and critical tone of this author's overview. Having also read through the METR report, I fee like frontier labs' pattern of getting PR about how impressive their models are "behind the scenes" has poisoned their ability to take security and reliability seriously. Sure these are new failure modes and the agents operate at a scale that's difficult to combat, but the lack of controls and concern for mitigating these sorts of hacks in the future is crazy to me.

The model for postmortems I was taught which has served me well in my career is thoroughly answering the following: - what happened / what was the timeline of events? - how was it mitigated and ultimately resolved? - what went well? - what went wrong? - where did we get "lucky" (meaning it could've gone worse but some arbitrary details about the incident worked out in our favor. usually stuff like "happened during business hours" or "we were already looking at a related thing that brought this to our attention before it was a worse outage") - (action items) how do we detect, mitigate, and prevent this type of failure in the future?

I really hope OpenAI has done an internal postmortem that answers these questions thoroughly. Most SWEs in the industry have to do such postmortems for much smaller outages with way less impact and risk of societal harm. This is probably another area where regulation and governmental oversight would help curb the risks. What's to stop OpenAI and other frontier labs from an intentional "accidental" attack that results in gaining access to competitors' systems?

I'm also curious what, if anything, Anthropic and Google have done differently to prevent a similar event. I suspect they actually monitored the agents as part of their studies and had better guardrails in their infra for how they set up their harnesses etc. for testing models, particularly when the other guardrails are absent as was the case here.

I read it and it is indeed quite good!

link for the lazy: https://gwern.net/doc/fiction/science-fiction/1995-egan.pdf

I can't wait to see what kinds of horrific AI-assisted surveillance these organizations end up getting access to. The only saving grace of these sorts of post-patriot-act police state surveillance systems was that they still had to be driven by a human. Imagine this stuff combined with an "agentic" workflow where the "human in the loop" is the dumbest and most aggressive personality imaginable working for the US Gestapo

I think if I'm honest about when I look at LLM output for code, it feels close to a code review of code written by another person, where I spend a decent amount of cognitive effort figuring out those similar small details in reverse, without knowledge or memory of the specific motivations. And furthermore, if I were to read every single line of code, word in a written work, and pixel or brush stroke in visual art of some kind, it would feel like an artificial understanding, rather than something I can fully sign my name to as a thing I understood the ins and outs of while creating it.

I fully get that some people don't care about this, and see themselves more as "designers", but I think those are the same sorts of people who if given a choice, would rather "build" by leading a team of people and claiming credit for "building" as part of their leadership, rather than by doing it themselves by hand and getting a sense of fulfillment from that effort. Also, "agentic" workflows feel a bit like giving up even the leading / designing aspects. With Claude specifically, I'm often impressed (albeit other times often frustrated too) by the small design choices it made while building the code that I hadn't explicitly asked for but end up preferring once I see the output. This to me indicates I didn't really build it

At the same time, I sense a lot of negativity (especially on HN) in comparison to Codex from OpenAI (who IIUC is bleeding money to keep this going, though maybe Anthropic is too) which didn't do as many promotional and nerf-ing shenanigans.

As somebody who doesn't want to constantly switch tools, is Codex really that much better, or is this sour grapes from folks who want to stay on the subsidized access plans and are mad at Anthropic's pricing model?

I suppose Nothing is carrying that torch forward but it's still disappointing to see. Even though most of it was extensions of Oppo tech and ideas into a US/Europe-friendly market position, it still felt like they were innovating and keeping Android ecosystem healthy and interesting beyond simple slab phones.

I was considering looking into a OnePlus phone as my next device for Lineage or Graphene OS, but I guess I'm glad I waited...

Another interesting finding which I've heard others corroborate is that even though the per-token cost was higher, it seemed to orchestrate the work efficiently and burn through fewer tokens, roughly evening out to approximately the same per-prompt token usage as Opus.

I suspect productivity will massively increase, the complexity and cognitive load of our work will similarly multiply, and yet we'll still being doing the now-more-complex work in some capacity for a similar number of hours.

My guess is that Google sees some kind of trend in a contingent of users preferring non-AI content and that surfacing AI content misleadingly has a negative effect on retention / watch time, and/or they're trying to get ahead of long-standing creators taking issue with the platform surfacing AI content disproportionately on account of it being excessively easier to upload in large quantities.

The much broader version of people getting a bit too full of themselves or trusting an LLM's sycophantic nature as validation that they are right or uniquely smart with their ideas seems to be the version I'm seeing more of in tech news sites and places like HN

[1] https://www.psychologytoday.com/us/blog/urban-survival/20250...

That said, I find it really hard to believe that they can't provide better specs and feature set for the cost. User-available memory of 3.5MB is incredibly low, especially with Python support. These could be really cool handheld computers if TI put more effort into their devices that already have a massive install base.

Currently, most of their popularity in my experience is "lock in" effect from teachers who are familiar with TI calculators and lab / curriculum materials that are specifically built around teaching through TI calculators. At this rate they're charging a lot and resting on their near monopoly status in education, which I'm sure is very profitable for TI.

There used to be a great app called WabbitEmu that emulated these devices on Android. I think they got a cease and desist but it was pretty neat to have back in the day

Can you elaborate on this? I thought that the Nazis were pretty obviously the "bad guys" due to committing genocide and mass casualties (combatant and civilian) while trying to expand their borders.

> It doesn't make any sense to try to judge morality based on casualty ratios.

Really, even the ratio of civilian casualties, or ratio of civilian casualties to combatant casualties? Those seem pretty relevant to morality in my book, but I might be misunderstanding.

Certain communications, especially technical writing, are "expensive" both in terms of the effort of the author(s), and in terms of the person-hours of people reading them to gain understanding. Like mission-critical code, they should be written and reviewed with care, and at the very least heavily edited from an automated LLM output to be unrecognizable as such.

I personally don't use LLMs at all in my designs and I remain skeptical of the value proposition for those who do.

Another obvious ChatGPT-ism. The fact that people are using AI to write these security posts doesn't surprise me, but the fact they use it to write a verbose article with spicy little snippets that LLMs seem to prefer does make it really hard to appreciate anything other than the simple facts in the article.

Yet another case in point for "do your own writing" (https://news.ycombinator.com/item?id=47573519)

I cannot think of a more important skill than surgery to continue training humans to do and to be wary of AI robotics replacing. Sure, some surgeries could likely be automated, but the entire point of specialist surgeons is to make choices and act in a timely manner in ambiguous situations with extremely high stakes.

What happens when the robot messes up? What happens when the internet is down, or the hospital is operating under abnormal circumstances? How do you teach, train, and collaborate with human medical workers and caregivers in a world where surgeons have been replaced by robots?

Most of the excess costs for healthcare and surgery aren't the humans doing the work. I think there's a lot of other areas we can optimize first, chief among those in healthcare being the cost structure around private businesses and insurers bloating the bill with administrative costs. There's a reason every other developed nation has a single-payer healthcare system and better outcomes, and I don't think an AI breakthrough is the only plausible solution to improving costs in the US. In fact, under the current system, an AI breakthrough in medicine would likely hurt the workforce more than it would improve costs.