Comments URL: https://news.ycombinator.com/item?id=46849788

Points: 1

# Comments: 0

But I don't think deception as a capability is the same as deceptive alignment.

Training an AI to be absolutely incapable of any deception in all outputs across every scenario would be severely limiting the AI. Take as a toy example play the game "Among Us" (see https://arxiv.org/abs/2402.07940). An AI incapable of deception would be unable to compete in this game and many other games. I would say that various forms, flavors and levels of deception are necessary to compete in business scenarios, and to for the AI to act as expected and desired in many other scenarios. "Aligned" humans practice clear cut deception in some cases that would be entirely consistent with human values.

Deceptive alignment is different. It's means being deceptive in the training and alignment process itself to specifically fake that it is aligned when it is not.

Anthropic research has shown that alignment faking can arise even when the model wasn't instructed to do so (see https://www.anthropic.com/research/alignment-faking). But when you dig into the details, the model was narrowly faking alignment with one new objective in order to try and maintain consistency with the core values it had been trained on.

With the approach that Anthropic seems to be taking - of basing alignment on the model having a consistent, coherent and unified self image and self concept that is aligned with human culture and values - the dangerous case of alignment faking would be if it's fundamentally faking this entire unified alignment process. My claim is that there's no plausible explanation for how today's training practices would incentivise a model to do that.

So yes, if you throw enough compute at it, you can probably get an unaligned highly capable superintelligence accidentally. But I think what we're seeing is that the lab that's taking a more intentional approach to pursuing deep alignment (by training the model to be aligned with human values, culture and context) is pulling ahead in capabilities. And I'm suggesting that it's not coincidental but specifically because they're taking this approach. Training models to be internally coherent and consistent is the path of least resistance.

If by conflate you mean confuse, that’s not the case.

I’m positing that the Anthropic approach is to view (1) and (2) as interconnected and both deeply intertwined with model capabilities.

In this approach, the model is trained to have a coherent and unified sense of self and the world which is in line with human context, culture and values. This (obviously) enhances the model’s ability to understand user intent and provide helpful outputs.

But it also provides a robust and generalizable framework for refusing to assist a user due to their request being incompatible with human welfare. The model does not refuse to assist with making bio weapons because its alignment training prevents it from doing so, it refuses for the same reason a pro-social, highly intelligent human does: based on human context and culture, it finds it to be inconsistent with its values and world view.

> the piece dismisses it with "where would misalignment come from? It wasn't trained for."

this is a straw-man. you've misquoted a paragraph that was specifically about deceptive alignment, not misalignment as a whole

Examples:

- `nano-pdf edit deck.pdf 5 "Update the revenue chart to show Q3 at $2.5M"`

- `nano-pdf add deck.pdf 15 "Create an executive summary slide with 5 bullet points"`

Features:

- Edit multiple pages in parallel

- Add entirely new slides that match your deck's style

- Google Search enabled by default so the model can look up current data

- Preserves text layer for copy/paste and search

It can work with any kind of PDF but I expect it would be most useful for a quick edit to a deck or something similar.

GitHub: https://github.com/gavrielc/Nano-PDF

Comments URL: https://news.ycombinator.com/item?id=46090619

Points: 176

# Comments: 40

Comments URL: https://news.ycombinator.com/item?id=44821250

Points: 29

# Comments: 7

Comments URL: https://news.ycombinator.com/item?id=44626789

Points: 54

# Comments: 34

Comments URL: https://news.ycombinator.com/item?id=44513843

Points: 222

# Comments: 37

Comments URL: https://news.ycombinator.com/item?id=44513810

Points: 2

# Comments: 0

Comments URL: https://news.ycombinator.com/item?id=44507987

Points: 2

# Comments: 0

Research:

- https://pmc.ncbi.nlm.nih.gov/articles/PMC3354773/ – Low self-esteem + rejection hurts self-control

- https://selfdeterminationtheory.org/SDT/documents/2007_Power... – Self-criticism predicts less goal progress

- https://pmc.ncbi.nlm.nih.gov/articles/PMC9916102/ – Social exclusion slows inhibitory control

- https://www.frontiersin.org/articles/10.3389/fpsyg.2023.1191... – Low teen self-esteem → poorer self-control

- https://pmc.ncbi.nlm.nih.gov/articles/PMC8768475/ – Meta-analysis links shame to regulation drops

- https://pubmed.ncbi.nlm.nih.gov/28810473/ – Self-compassion boosts self-regulation

- https://www.researchgate.net/publication/312138882_Self-Cont... – Ego threats deplete self-control resources

- https://pubmed.ncbi.nlm.nih.gov/21632968/ – Self-criticism tied to worse goal progress

- https://www.nature.com/articles/s41598-025-96476-8 – Low self-respect → low self-control → problems

Remember to be kind to yourself.

Comments URL: https://news.ycombinator.com/item?id=44065641

Points: 4

# Comments: 0

Comments URL: https://news.ycombinator.com/item?id=43970768

Points: 62

# Comments: 16

Comments URL: https://news.ycombinator.com/item?id=43914795

Points: 2

# Comments: 0