Sharing a diff of the system prompts here: https://twelvetables.blog/comparing-claude-fable-5s-system-p...

The big difference is that the system prompt has a whole section dedicated to directing Fable how to communicate with users, and give them greater information about the (assumedly long-horizon) tasks it has completed.

Comments URL: https://news.ycombinator.com/item?id=48465813

Points: 3

# Comments: 0

I think the way the paper lays out the performance regimes is pretty interesting, but I don't think they achieved their goal of demonstrating that LRMs can't use reasoning to solve complex puzzles organically (without contamination/memorization): IMO testing the model's ability to define an algorithm to solve the puzzle would have been a better evaluation of that (rather than having the model walk through all of the steps manually). I don't know that I'd use an LRM for this sort of long-tail reasoning where it has to follow one single process for a long time over just one prompt; if I needed a really long chain of reasoning I'd use an agent or workflow.

It sounds more like the tests measure a model's ability to reason coherently and consistently over many steps rather than a model's ability to understand and solve a complex puzzle. For example, for the Tower of Hanoi, a prompt like "Define an algorithm that will find the sequence of moves to transform the initial configuration into the goal configuration" (e.g. "find an arithmetic series formula, young Gauss") seems like it would have been a better approach than "Find the sequence of moves to transform the initial configuration into the goal configuration" (e.g. "add up all these numbers"). This is kind of seen in how the study included a step where the LRMs were given the algorithm and then asked to solve the problem, the focus was on an LRM's ability to follow the steps, not their ability to come up with an algorithm/solution on their own.

In a job interview, for example, who among us would accept inability to hold all of the `(2^n) - 1` steps of the Tower of Hanoi in our brain as evidence of poor reasoning ability?

Again, I think it's a really interesting study covering a model's ability to consistently follow a simple process over time in pursuit of a static objective (and perhaps a useful benchmark moving forward), but I'm not confident that it successfully demonstrates a meaninful deficiency in overall reasoning capability.

[1]: https://www.americanscientist.org/article/gausss-day-of-reck...

It parses ldapsearch or dsquery output and writes markdown files with some extra logic added in so you can visualize nested memberships when you open the folder with Obsidian and use the graph view--it's kind of like a very low-powered Bloodhound (https://github.com/SpecterOps/BloodHound) in that sense, but it is also much quieter. It also parses `useraccountcontrol` and some of the logon-relevant attributes in AD to automatically tag accounts that are particularly interesting or uninteresting to attackers.

Some core features: - Parses LDAP attributes from text files into structured Obsidian markdown - Intelligently categorizes objects as Users, Groups, or Computers - Automatically identifies administrators and administrative privileges - Detects potentially risky account configurations (stale accounts, low logon counts) - Creates Obsidian links between related objects to enable network visualization - Processes UserAccountControl (UAC) values with explanations - Converts Windows timestamps to human-readable format - Smart append mode that only adds new data to existing files - Tagging for easy filtering and searching in Obsidian

It's not novel, but it's been quite useful for me in a few situations.

Comments URL: https://news.ycombinator.com/item?id=43568958

Points: 3

# Comments: 0

It's been useful for me in thinking about security, and I think it could be helpful for others.