These tools have only been in use for a short time and the current harnesses/system prompts are quite limited. Claude code is mostly limited to your codebase where you have version control. Excel is different.

I foresee that once people hand over more power to full agents there will be some nasty surprises. Im sure there will eventually be demand for some kind of limits

For example: imagine having just untrusted content and private data (2/3 parts of the trifecta). The untrusted content can use a "Disregard that!" attack to cause the LLM to falsely modify the private data. So I think the whole "trifecta" is not necessary and the key thing is that you simply can't have untrusted stuff in your context window at any point.

I mention in the footnotes that I think that it makes more sense for the end-user of the LLM to be the one running it. That meshes with RBAC better (the user's LLM session only has the perms the user is actually entitled to) and doesn't devolve into praying the LLM says on-task.

But I am still unsure that it actually is robust. I feel like you're still vulnerable to Disregard That in that you may find that the model just starts to ignore your instruction in favour of stuff inside the context window.

An example where OpenAI have this problem: they ultimately train in a certain content policy. But people quite often bully or trick chat.openai.com into saying things that go against that content policy. For example they say "it's hypothetical" or "just for a thought experiment" and you can see the principle there, I hope. Training-in your preferences doesn't seem robust in the general sense.

Comments URL: https://news.ycombinator.com/item?id=47439393

Points: 1

# Comments: 0

I feel like most of the opinions about FreeCAD online are out of date, since at least 1.0 if not later.

I mainly use it for planning things to make out of wood or print out of plastic.

cf this programme from the BBC Archive: https://www.youtube.com/watch?v=vvXpvH99tic It was clearly badly designed and problematic right from the start with little in the way of shops or services for the residents. But that's the thing about monumentalist architecture: looks singular but function is an afterthought. The south bank centre is similar.

I lived locally for 10 years and visited only a handful of times. Mostly it was just an obstacle in itself: it creates a lot of level changes (read: steps) and moving around it on foot or by bike is annoying.

There are already some measures for young people, like the 6 point thing. Maybe there could be more. Doesn't change the facts about dangerous OAP drivers

I'm talking about removing licences due to cognitive decline. It's not a temporary condition

Politically very difficult to take people's licences away though, especially when it's permanent, not their fault and it makes their life a lot worse.

A significant number of anki users (eg: medicine, law - others) are working with pre-made decks and if you look at anki's competition - all of them offer pre-made decks as a key part of it. Medics have always used flashcards (many university bookshops sell physical flashcards for medics) and I don't each medical student would benefit from producing, eg, their own anatomy flashcards.

I would substitute "the material" for "the cards" in this sentence. Making the cards yourself is one way to do that, but it's not always the most time effective - imagine the extra work put onto a medical student having to make the cards for every subject they need to cover. That is what ankihub does and it seems to be very popular

But yeah: downloading the median deck off of ankiweb: very sub-optimal

Commercial decks, where the deck maintainer is paid for his efforts, make a lot of sense.

And I suppose if they are making money out of the ecosystem, it also only makes sense that commercial deck makers make a contribution to the technology that makes it possible. I suppose I would prefer that be a contribution rather than ownership and custody, but I suppose Anki's license terms (it is AGPL3+ - I think without a CLA) prevents them closing it.

So cautiously optimistic

Comments URL: https://news.ycombinator.com/item?id=46731579

Points: 3

# Comments: 0