My main takeaway from that interview was that tier1 CS is a low-wage, high-turnover industry, so the amount of training agents can realistically get is limited. AI support is cheaper, smarter, and can take in more context faster. With AI, it makes sense to have a "support engineer" who identifies the edge cases and writes Markdown documents instructing agents on what to do in those cases. This actually lets you make agents much, much more helpful than human employees ever could be.

[1] (There's a high-quality transcript available) https://www.complexsystemspodcast.com/episodes/des-traynor/

If you ban or censor a book, you immediately make the book seem more valuable. Because governments aren't omnipotent and all restrictions can be overcome (see the war on drugs as a particularly recent and pertinent example), you just Streisand-effect yourself.

If, on the other hand, you take away the popularity and social status of those who read that book, branding them as gullible idiots in the popular imagination, people will have an aversion to reading it. You don't need to ban access to the book, in fact you shouldn't do that, just make sure that talking about it will get people to lose all their friends.

Social media are the modern arbiters of popularity. If social media bans a subject, people get angry. If it just quietly deboosts anybody who talks about that subject, "well let's better not do that, we tried and people really didn't like those videos I guess"

His most important projects are ffmpeg (codec specs), qEmu (ISA specs), QuickJS (the EcmaScript spec), tinyC (the C spec), and his telecom company (LTE specs). I guess the pi calculations and neural network stuff are exceptions.

Just to be clear, this doesn't make his work any less impressive. Highly performant codec and emulator implementations are no easy feat; it's just interesting that most of this work falls into that relatively narrow area.

Recent LaTeX versions started to be able to do it, it'd be a shame if people started switching over just when LaTeX finally started pulling their accessibility act together.

1. How does Iroh handle key rotation / leakage? Could you build some kind of hot/cold system on top of it, where you'd have a cold "identity key" in airgapped, secure storage, used only to issue certificates for your hot "traffic acceptance" key?

2. Is there any kind of peer discovery / DHT, either built-in directly or through some semi-official higher-level protocol, like DNS for IP?

3. What about human-friendly peer names? Those are almost required for end-user friendly applications. Most solutions of that problem either assume that every single user is willing to dedicate their life to configuring DNS, rely on a trusted third party, or delegate the responsibility to a blockchain.

4. What are the channel reliability properties, and are they configurable? Can you decide how to handle out-of-order or lost packets, or does the protocol enforce a decision? If you're willing to tolerate loss, duplication and reordering, can you avoid head-of-line blocking?

5. Is peer anonymity a goal?

6. What about two mostly-offline peers who wish to communicate (think smartphone apps that can't be connected 24/7 due to battery concerns)?

Overall, cool project.

I'd personally love a "Homebrew light", compatible with most of the ecosystem (sans git taps, Ruby formulas and binary packages), written in a language with fast startup times, not keeping unnecessary files on disk, with support for parallel downloads, and terminology that is much easier for the newcomers to remember and keep straight.

React already existed in 2016, so did Vue and Typescript. Never mind good old JS.

I frankly can't imagine a device capable of supporting modern TLS stacks but incapable of supporting JS. Much less a phone, which in many countries basically requires LTE (and sometimes even VO LTE) support to function at all, due to the 2G and 3G shutdowns.

Does it, though?

Does Amazon have a clause in their contracts that forbids data sharing with any and all third parties? Is all AWS support and datacenter personnel employed directly by Amazon? Do they seriously have no third-party contractors?

How many such devices can still support modern TLS certificates anyway? By this logic, shouldn't we also use plain HTTP instead of TLS?

There are many people who don't live in a country where English is spoken natively, but who speak it well enough to lurk on the English internet. Those people are exposed to American and British politics and start to form opinions. It's not unusual for us to have our own takes on what happens in these countries.

Microsoft's approach to data is basically "we promise nobody else but you and your government can access it, we can but we pinky swear we won't." This promise is mostly enforced at the legal layer and through legal consequences, not technical safeguards. If they think they can get away with it (or are forced to get away with it by the US government), there's nothing stopping them from using your data in whatever way they want.

When they can, Apple designs their systems so that they physically don't even have the capability to use your data, even if it's processed on their own servers. They're not privacy maximalists like Signal is, they care more about user experience, but they do aim for the highest level of privacy you can get while still having a good experience, and when they do need to make sacrifices, they typically let you opt into the privacy features if you really want to.

I'm far more inclined to believe that Microsoft is secretly (or not so secretly) collaborating with the US government than that Apple is.

When you are writing an essay and realize midway through a sentence that what you've written doesn't make sense, you go back and edit. An LLM can't do that, the only thing it can do is keep on generating. Because training data typically contains full essays and not half-finished sentences which were then edited, LLMs have a strong preference for "saving face" and producing grammatically correct, internally coherent outputs. They will often do so even if the only way to write themselves out of the corner they wrote themselves into is to lie. To maintain internal coherence, they'll then repeat that lie for the rest of the response.

This is also why changing response structure used to affect LLM performance so dramatically. If you asked an LLM to solve a math problem and all-but-forced it to start with the answer, it would have had to calculate that answer before emitting any tokens, something which it very often wasn't able to do. If it was told to follow up the answer with an explanation, it would produce a plausible-sounding explanation to maintain coherence.

If, on the other hand, it was told to start by "thinking step by step", it would often be able to solve the first step, and then the next one given the results of the first, and so on, until it was able to reach the answer. Because the answer came last, it wasn't committing to anything, so had no reason to "save face" and lie.

This part of the problem is basically solved now with reasoning; reasoning is where all the step-by-step stuff happens, even if users aren't always able to see it. In the process of RLVR, models even train themselves into outputting phrases like "let me check my answer once again" in the chain-of-thought; those serve as their "life rafts" which they can use to both save face and change their answer.

I find it a nice middle ground between the craziness of vim and the slowness of traditional cursor usage.

source available - whether you can read the code

open source - whether you can run (a modified version of) the code on some piece of hardware you own

open hardware - whether the hardware they sell you lets you run modified versions of their code

open contribution - whether they want your modifications

free software - whether your modifications have to be open source too

If it's at least source available, it can have any combination of these.

With humans, it's acceptable to have an "authenticate a customer" tool and a "reset the customer's password" tool as two separate applications. You can put in the manual that the latter can only be used after the former.

With agents, you can achieve the same outcome, but the constraint needs to be enforced by code, not job training and employee handbooks.

And this is where most of the vulnerabilities come FROM, AI or no AI.

You can't expect entry-level support workers to be responsible. Either you codify a process (which you can still do with an AI), or you become like a cell carrier, extremely vulnerable to SIM swapping attacks.

This is exactly what I mean; if you give your agent access to some knowledge base through RAG; you should assume that this knowledge is now public information. If you don't want it to leak, design your agent so that it doesn't have access to it.

This framing doesn't consider context poisoning attacks, on which much has been written already and which merit their own defenses.

I was surprised how well it worked, even then.