The issue is not in the verification system, but in putting quantifiable bounds on your answer set. If I ask an LLM to multiply large numbers together I can also very easily verify the generated answer by topping it with a deterministic function.

I.e. rather than hoping that an LLM can accurately multiply two 10 digit numbers, I have a much easier (and verified) solution by instead asking it to perform this calculation using python and reading me the output

I’m still not sure why some of us are so convinced there isn’t an answer to properly verifying LLM output. In so many circumstances, having output pushed 90-95% of the way is very easily pushed to 100% by topping off with a deterministic system.

Do I depend on an LLM to perform 8 digit multiplication? Absolutely not, because like you say, I can’t verify the correctness that would drive the statistics of whatever answer it spits out. But why can’t I ask an LLM to write the python code to perform the same calculation and read me its output?

> I think it follows that we should not use llms for anything critical.

While we are at it I think we should also institute an IQ threshold for employees to contribute to or operate around critical systems. If we can’t be sure to an absolute degree that they will not make a mistake, then there is no purpose to using them. All of their work will simply need to be double checked and verified anyway.

Comments URL: https://news.ycombinator.com/item?id=40311974

Points: 4

# Comments: 0

The only thing separating a talking parrot and humans is our accuracy in shaping our words to the context in which they’re spoken.

Sure it’s easy to liken a low resource model to a talking parrot, the output seems no better than selective repetition of training data. But is that really so different from a baby whose first words are mimics from the environment around them?

I would argue that as we learn language we implicitly develop the neural circuitry to continue to improve our lexical outputs, this circuitry being concepts like foresight, reasoning, emotion, logic, etc and that while we can take explicit action to teach these ideas, they naturally develop in isolation as well.

I don’t think language models, especially at scale, are much different. They would seem to similarly acquire implicit circuitry like we do as they are exposed to more data. As I see it, the main difference in what exactly that circuitry accomplishes and looks like in final output has more to do with the limited styles of data we can provide and the limitations of fine tuning we can apply on top.

Humans would seem to share a lot in common with talking parrots, we just have a lot more capable hardware to select what we repeat.

Always see parallels drawn between Obsidian note structures and whole “2nd brain” idea for personal knowledge management, had seemed like a natural next step would be to implement note retrieval for intelligent references. Will have to check this out