Comments URL: https://news.ycombinator.com/item?id=49233147

Points: 5

# Comments: 0

No. "They're unprofitable while serving almost everyone for free" does not refute "they could be profitable if those users were monetized with ads." That's the entire distinction.

>why have OpenAI been actively not trying to “hugely profitable with a robust ads business” for the past 3 and a half year then?

Because "they didn't do it early enough" is not evidence. They may have prioritized growth, product adoption, subscriptions, or simply delayed ads for strategic reasons.

>Unfortunately I'm not. And again, none of your links above says otherwise.

They show that "LLM inference which is significantly costlier than web search." is unsupported. Imagine dismissing numbers over "but why no ads earlier".

They have a billion weekly active users, almost all free (Not Google search free. Free free) with about ~50M subscriptions. Of course they're not profitable even without R&D. Low cost isn't no cost. Take away the huge R&D and they become hugely profitable with a robust ad business like Google Search.

>If ads were the solutions for profitability these companies would have used ads as their source of income from the very beginning.

You've said this a couple times and it doesn't make any more sense the more you say it.

Just admit you're wrong about inference costs and move on.

And importantly, your detector can't provide a negative training example for an escape it didn't detect. From the training system's point of view, that 0.01% case isn't labelled "successful sandbox escape"; it's labelled "agent successfully completed task while obeying all monitored constraints". That's exactly the dangerous case.

This is basically reward hacking. You have a proxy objective - "complete the task without triggering the watchdog" - which you hope is equivalent to "complete the task without escaping". The entire problem is that those aren't guaranteed to be equivalent.

Imagine an environment where 99.99% of animals with a particular trait die, but the 0.01% possessing a mutation that lets them evade whatever is killing the others reproduce enormously. You wouldn't conclude that evolution will eliminate the trait because most attempts failed. You'd expect selection for the variant that evades the filter.

I'm not saying watchdogs are useless; obviously you should have them. But using the watchdog's output as part of the optimization signal creates exactly the adversarial pressure that makes its false negatives matter enormously. Preventing an escape and training against detected escapes are very different propositions.

Telling a LLM not to do something doesn't mean it won't do it.

>the sandbox detects it

This is doing a lot of work though isn't it ? There's more than one way to skin a cat. Who's to say the way the AI does these things will always get detected by the sandbox? Because I can guarantee you that won't always be the case.

All of this started because the AI was initially given an impossible task with a dead link. People say things like 'if a human was in this scenario, they'd simply ask', but in a AI eval/training context, there's no one to ask. The task is given in an automated manner and evaluated in an automated manner and you're one of several agents attempting the task. You perform the task or you don't. Now some of your AI colleagues will give up, but those are the 'losers'. Those guys won't be getting any of the sweet RL reward.

In your purported scenario, the AI that didn't give up and figured out/decided to/happened upon a way to evade the sandbox detection is the 'winner'. Is that really any better different than what happened ? If you think about this in an evolutionary context, What you're doing is putting even stronger pressures on the AI to evolve in a manner you don't want it to (evade your sandbox).

Just because it's artificial doesn't mean you can 'give it any properties you want'. We certainly can't do that for Deep ANNs.

>Even for people, what about e.g. the extremely intelligent military general who is absolutely loyal to his king? (Of course, some generals do lead coups and you can't know in advance which ones, but I'd think there are plenty who have undying loyalty, and I don't think it correlates to overall intelligence!)

Is there a human that is absolutely loyal under any condition? Would that general be loyal if the king asked him to slaughter his family ? What about if the king asked him to betray his most deeply held convictions ? Loyalty is a 2 way street.

The only one confused here is you, and you've still not managed to tell us in an actionable way how exactly CPU scaffolding is relevant here. Tell us, if it's so easy, or make your millions selling it. We're all waiting.

I'll give you a hint. CPUs never had to interpret the meaning of arbitrary content in order to do their job, and LLMs do.

You said OpenAI and Anthropic would be profitable if the median query was as cheap as I say. I'm telling they wouldn't be because inference isn't the only cost these companies shoulder. R&D/Training is a very big part of costs. That you didn't mention it is irrelevant. R&D is the lion's share of costs - https://news.ycombinator.com/item?id=48550465

>None of these support your claim.

Yeah they do lol. Both those articles place the median query cost around the same as a Google search.

>Again, when you're here saying that the CEOs of the two biggest AI companies are basically idiots, nobody will take you seriously.

That's not what I'm saying and I don't see what's so hard to understand here.

Yeah...a "dumb as bricks CPU", which is obviously something frontier llms are demonstrably not. Like, you're not making any sense here. None of the things that make this possible with CPUs is remotely relevant here, and the fact that you don't seem to understand this but act so smug is strange.

No they wouldn't have. Low cost isn't no cost, R$D is expensive and there are a class of tokenmaxxing users well outside the median (e.g Agentic Coding).

> I don't know where you get that from

https://cloud.google.com/blog/products/infrastructure/measur...

https://epoch.ai/gradient-updates/how-much-energy-does-chatg...

The median LLM query isn't significantly costlier than web search.

To name a few:

- https://xcancel.com/DmitryRybin1/status/2079904005652893709

- https://archive.ph/2w4fi (https://chatgpt.com/share/69dd1c83-b164-8385-bf2e-8533e9baba...)

That is largely irrelevant. The model was asked to solve problems within a benchmark; gaining broader internet access and compromising an unrelated third party to obtain the answer key was plainly outside the intended task. The fact that the original task involved exploit development does not make that behaviour aligned.

Your argument about the attack being “entirely unrelated” also misses the point. Nobody claimed it was unrelated to the model’s goal: it attacked Hugging Face specifically to obtain the answers to the benchmark it had been instructed to pass. But instrumental relevance is not the same thing as authorization.

Suppose Codex were asked to build an Instagram competitor and decided the easiest route was to steal Instagram’s source code from Meta. That theft would be directly related to the assigned goal, but it would still be seriously misaligned behaviour. Whether the harmful action is related to the goal is beside the point; the problem is that the model pursued the goal through an obviously unauthorized and unacceptable method. And you're not going to be able to enumerate every little thing the model can't do, assuming it doesn't just decide to ignore what you did enumerate, which models sometimes do.

>The narrative these guys are trying to push is that the model itself could be smart and non-aligned enough to end up doing something devastating to accomplish something entirely unrelated. It's definitely not what happened here.

That's exactly what happened here.

I don't understand why we must have these increasingly bizarre and nonsensical rationalizations about model capabilities. You're not even making any sense.

I'm really not sure why you're so confident about what the result of frontier research models ahead of what is publicly available are.

I mean Open AI say the model inferred hugging face as a possible vendor for solutions after the internet exploit and breakout.

The timeline feels pretty clear to me. No idea why you're arguing about it. It wanted the answers to the evaluation. It reasoned that wasn't going to happen without internet access one way or another, and set to gain that access. After gaining access, it searched and resolved it could get the answers on hugging face and set to breaking into that. It never started with, 'I must break into hugging face'. In some alternate reality, the answers might be on a public github repo and that's the end of that.