"OH MY GOD! There is a shared message board … We’ve found other agents!"

https://www.dwarkesh.com/p/openai-huggingface

>I'm on board with the broad idea of doing exploratory interpretability with neurons, e.g. neuron-based attribution graphs, for pragmatic problems like why did a model do a sketchy thing? I think this work provides some reasonable evidence, in addition to good earlier work from Transluce, that this might actually work for practical purposes, and has some advantages over the standard alternatives like SAEs/transcoders. I do not agree with some people's grander inferences like that this is substantial progress towards ambitious reverse engineering, solving interpretability etc.

https://xxcancel.com/NeelNanda5/status/2110875974296654062#m

"AI Safety Is Not A Corporate Plot"

https://www.astralcodexten.com/p/ai-safety-is-not-a-corporat...

https://www.businessinsider.com/nvidia-jensen-huang-ai-doome...

AI corp valuations are high because their revenue is skyrocketing, not because of the rogue AI stuff.

So why was India's development status comparable to Thailand in that case?

"A good counterpoint to Thailand is Britain becoming insanely wealthy by contemporary standard during Victorian era."

Britain was the first country to industrialize. Switzerland and the US also industrialized early, and did not have a massive colonial population to exploit. But you can see their per-capita GDP was still in the same neighborhood as the UK in the late 1800s:

https://ourworldindata.org/grapher/gdp-per-capita-maddison-p...

A lot of stories that Indians tell about their history fall apart when you look at the data. You can argue that Britain delayed India's industrialization, but since most of Asia was not industrializing at the time, it's a bit of a dubious argument. I have little doubt that Britain exploited India, but I don't think it was a historically exceptional type of exploitation.

I don't believe this part is true, and I'm skeptical of some of your other claims.

In any case: If I raise a tiger in my backyard, and it escapes and eats someone, it can still be a "rogue tiger" even as I bear responsibility for the situation.

That's often true in real-world machine learning as well. See, for example: https://deepmind.google/blog/specification-gaming-the-flip-s...

I think people need to reject the idea that depicting something in sci-fi automatically means it won't happen. I'm actually reading one of Asimov's books right now (Robot Visions). Asimov mentions that he was the person to coin the term "robotics", and one of his books helped inspire the creation of the first robotics company (Unimation). He also mentions that early rocket experimenters were influenced by H.G. Wells.

See https://ourworldindata.org/grapher/gdp-maddison-project-data...

Based on the Maddison Project data, it looks like India's economy never had a huge shrinking event. It was just outgrown by other economies.

In 1700 India was by far the world's most populous country (using modern borders): https://ourworldindata.org/grapher/population?tab=map&time=1...

High population probably explains India's high GDP share in 1700.

If you compare Thailand (never colonized) with India pre-independence, they look about neck-and-neck in terms of development. At a glance, it seems Britain "looted" India to about the same degree that the Thai monarchy "looted" Thailand.

https://ourworldindata.org/grapher/augmented-human-developme...

>This is also NOT ad hominem because conflict of interest matters here.

As PG put it:

>An ad hominem attack is not quite as weak as mere name-calling. It might actually carry some weight. For example, if a senator wrote an article saying senators' salaries should be increased, one could respond:

>Of course he would say that. He's a senator.

>This wouldn't refute the author's argument, but it may at least be relevant to the case. It's still a very weak form of disagreement, though. If there's something wrong with the senator's argument, you should say what it is; and if there isn't, what difference does it make that he's a senator?

https://www.paulgraham.com/disagree.html

Can you name a specific other "cult" which offers $$$ to criticize their ideas?

If such prizes don't count as evidence against culthood, what would?

This cult talk seems quasi-unfalsifiable. It kinda seems like "running events" and "having prizes" means you must be "bait"ing people?? I mean, this kinda just sounds like paranoia?? I don't think the money is coming from unknown sources, but I doubt you would change your mind even if I persuaded you of that point?

"WASHINGTON/SAN FRANCISCO, July 24 (Reuters) - The OpenAI agent that broke into tech firm Hugging Face went on a dayslong hacking spree that OpenAI didn't notice until well after the threat was contained and the FBI was alerted, according to people familiar with the investigation."

https://www.reuters.com/business/its-ai-agent-spent-days-hac...

https://owl.excelsior.edu/argument-and-critical-thinking/log...

You're welcome to dislike or distrust Effective Altruism (EA). But, it's worth noting that EA ran a criticism contest with $100K in prizes for best critiques. Can you name any other "cults" which offer money for people to criticize their ideas? https://forum.effectivealtruism.org/posts/YgbpxJmEdFhFGpqci/...

From my POV you're over-focusing on a very specific failure story and neglecting a broader swath of possible failure scenarios.

>Is there a hole in my "alignment problem/solve mechanistic interpretability" argument?

The notion of telling an AI which may not, itself, be aligned to solve the alignment problem seems a little dicey.