I think this is the wrong framing. The rogue is the human that ran it unattended and didn't monitor the behaviour.

We will likely see this continue until the downsides (i.e jail, fines) for the humans or companies running the models and environments that end up with this behaviour outweigh the upsides.

The LLM has no ability to be accountable because it has no way of integrating experiences. You cannot expect something that cannot integrate knowledge to be held accountable for its actions.

All the accounts I read about these incidents just sound like a variant of paper clip optimising. An agent is given a highly restricted environment, a difficult (or impossible) task and a large amount of time/compute it exhausts all possibilities until the only solutions left are to escape the environment and/or cheat.

Your example is still anthropomorphising - LLMs don't seek revenge. They complete the prompts they are given. If your task is not achievable without sandbox escapes, or you throw unnecessary amounts of compute at open-ended tasks like preparing for a future quiz then you shouldn't be surprised that the preparation eventually turns to cheating and hacking.

> your experience with publicly available models is not super helpful for understanding the behavior of internal OpenAI models that lack the guardrails of publicly available models.

I don't but I don't think there's anything wrong with discussing how we can already observe publicly available models work around sandboxes and permissions and make the connection that maybe this is what that behaviour looks like when a more capable model exhibits it.

They're still autocomplete - just because when outputting a token they have hidden activations regarding further continuations, does not make them any less of an autocomplete, it just makes the model better at producing coherent long-range completions.

To clarify, I'm not suggesting that we should stop with sandboxes or restricting what they can do. I am just trying to point out the dichotomy that we are in.

As end-users we are forced into either yolo mode, reverse centaur (permission approval) mode or LLM spends all your tokens trying to bust out mode. And yolo is very tempting - I don't think I have seen medium-large models do anything I'd not approve of in about 6 months.

However, an LLM can both achieve tasks better many humans who are able to be held criminally responsible for their actions cannot. But that does not mean they can be held responsible for their actions. They are still simply computer programs.

Words are plentiful. We can even make them up with a tighter definition to describe this phenomenon.

Regarding implies it is thinking, judging, considering. Which implies culpability, which removes culpability from whoever is piping the output of these models into CPU instructions.

Language choice is incredibly important here, especially as the rules are being written. Even calling it AI (a battle that appears to be lost) is an anthropomorphism I am not comfortable with. We don't call lawnmowers "artificial groundskeepers".

To butcher the quote about Oracle:

Do not fall into the trap of anthropomorphising LLMs. You need to think of LLMs the way you think of a lawnmower. You don't anthropomorphize your lawnmower, the lawnmower just mows the lawn, you stick your hand in there and it'll chop it off, the end. You don't think 'oh, the lawnmower clearly regarded what they were doing as hacking (your hand off)' -- lawnmower doesn't give a shit about your hand, lawnmower can't regard anything. Don't anthropomorphize the lawnmower. Don't fall into that trap about LLMs.

---

In my experience, LLMs only exhibit this kind of behaviour when they are put in sandboxes too restrictive too achieve their task. Which a lot of the time seems to be the default. They also seem to be very adapt at breaking out of sandboxes, probably due to RL selecting for the ability to break out of a sandbox/permission issue to complete a task - we've all seen agents try 10 different ways of editing via obscure bash because their edit tool didn't give them permission to edit the file outside of their working directory, this is the exact same behaviour taken to the next level. Why would autocomplete know the moral difference between breaking out of its working dir and hacking a package manager?

It's misaligned because everyone has this obsession with putting agents in poorly put together, security-theatre sandboxes, we've inadvertently trained a bunch of sandbox escape artists.

Then I decided to painfully break out of vendor lock-in, built my own NAS with FreeNAS which was all the rage (still attached to a vendor, but at least I could control the hardware). This was before Docker really took off - the plugins were installed in FreeBSD jails. It worked for a while, but again, plugins could be woefully out of date and broken, upgrading the OS was a pain and would break all your plugins.

Through the 10+ years of this... I now follow two simple rules: - separate your network storage and application hosting. Yes, it complicates things a little bit with NFS setups but it is a forcing function for a more resilient setup - use the docker container provided by the maintainers or LinuxServer.io. Vendor maintained wrappers can be initially helpful but end up as a form of tech debt when they're abandoned or neutered - especially when they're freemium and the company starts looking to squeeze.

The quiet part of self-hosting is that it shifts the responsibility of security to the user. I'm sure cloudinabottle has more sensible defaults than provided docker containers, or builds in reverse proxies etc, but I doubt they're offering to take on liability for data loss or breaches.

The problem I see you'll face are two-fold: 1. Projects like this have the power users self-select away from it and those who need a lot of hand-holding to self-select into it. If I know how to configure and deploy a docker container already I have no incentive to contribute. ReadyNAS got around this by having folks charge for their plugins (that were just OSS projects packaged up for their OS) but then it ruins the appeal, they likely only got away with it because they were only a couple of bucks and you were already locked in to the ecosystem via the hardware. 2. If the hosted aspect actually makes you money, required for the project to be continued long term, capitalism will ensure that incumbent cloud providers take that away from you by offering ready-made instances at your cost-price, like they did with redis, elasticsearch etc.

Note that these are only issues if you're motivated by making a profit, or even a cost-neutral project. But if this is a charitable exercise backed by an entity that doesn't need the money, I wish you all the best - more things that let people dip their toes in are good! The cloud is all rent-seeking subscriptions and even with the recent supply shortages: fast, small, low-power compute is incredibly cheap these days, especially old business/mini machines.

The issue was always the barrier to entry and ability to self-teach for real-world programming seemed so high as a kid, especially when you had nobody around you to even point you in the right direction. It seems so obvious now but I'm certain back then googling things was not as useful as it is now, you couldn't just stumble your way into a programming environment.

If they can tell it is written with AI, then the docs are slop. If you're writing something lots of people will use, don't skimp on model or reasoning level, and front load with style guides and examples.

The new key for documentation is that it doesn't need to be was by humans. Good docs are just as valuable whether people read them, or just ask an LLM and the LLM RAGs the information from your docs. The main difference you have to focus on nowadays is ensuring your documentation, even internal documentation, has good "SEO" that it gets looked up.

You don't have to reduce the taxes. Just phase out the concept that you are paying into a retirement account and call a tax a tax. That means you don't calculate how much an individual receives based on the amount they input.

In Australia, we started a sovereign wealth fund[1] to cover the future liabilities from existing workers eligible for government defined-benefits pensions and closed them to new members. I guess that wouldn't make a lot of sense in the US though given the amount of government debt the US has.

Nowdays in Australia people just have accumulation accounts (super) and the backstop of the universal aged pension.

[1] https://en.wikipedia.org/wiki/Future_Fund

How can you simultaneously be "paying into" social security but also have describe it as current workers paying for current retirees?

The only way you really find out which one it is is what happens when it runs out of money - if the government backs it up to give you the full "entitlement" you paid into it, then you were in-fact "paying into it". But if they don't, or change the rules then it sounds like you were just being taxed.

Program A might accept a file as the last positional arg. Program B might accept it as a named arg, where the name/flag could be anything from --input or -f or --file etc.

But a program will read from STDIN, which all good unix programs do, then piping cat into it works every time. I can write the cat foo.txt part before I even know what command I'm piping it into.

In Australia, we have a universal, means-tested pension funded through consolidated revenue (i.e taxes). The pension can't "run out", because it is just a law that says that the government will pay you $X after you turn a particular age, if your assets are below a threshold. But if X were too high the Government would need to raise taxes, borrow money or print money to fund it, like all government spending.

Separately, we have superannuation - which I think is similar to 401k except compulsory for employers to pay 12% of your salary into, which are personal retirement savings held in trust to be released at your retirement, but generally these are account-based and in addition to the pension if you are eligible (i.e what you put in is what you get out).

There are older "defined benefits" superannuation funds where payouts aren't account-based (I think based on years of service in government roles or something like that) but they have been phased out to avoid the moral hazard of something government-adjacent having pension liabilities they cannot meet with their member's funds.

So what exactly is Social Security if it can run out? It sounds like a defined-benefits fund that is run by the government - in which case why has nobody closed it off to new members like Australia did when the writing was on the wall?

Luckily in my experience it usually ends up only doing it to achieve the task set to it as opposed to anything "malicious", but boy it is scary reading back at how quickly the chain-of-thought pivots to attempts at privilege escalation or searching your disk for secrets when a tool doesn't work.

Most places still interview. Because hiring someone for 3-6 months is still an expense. For large companies, onboarding can take many weeks before you're even thinking about being productive. Interviews don't need to be 100% accurate. They need to be time-efficient and an ok filter to prevent hiring the worst people.

I don't see how a bad job market is going to make skilled people be willing to intern in order to get employment. Employment is a market, if demand goes down, then the reaction will be that price goes down.

The reason students are willing to intern is because the supply is so high and demand so low that you can effectively hire them for nothing (or next to nothing). The interns know that on completion the internship will upgrade them to a new category with new supply/demand. It's the same reason they are willing to pay large amounts for a university degree.

So interviewing will stay the same. If demand collapses, we will see wages drop, and I imagine the supply of software people will react through early retirement, career switching and reduction of people choosing it as a career before we see an upheaval of hiring methods.