With that argument, the basic calculator would have been impossible.

It's pretty clear that machines can be better than the best humans at memorizing things, computing relationships between concepts quickly, etc. So while you can argue that LLM are not the path forward, I have no problem imagining that using all of humanity's knowledge with a machine could beat the best human at a field.

Sure, there are some software that have similar "can't have bugs" requirements. I imagine the computers on Moon missions also had that kind of high bar. I wouldn't use NASA requirements as a proof for how LLMs should be used.

In particular, I found this very misleading or irrelevant:

a typical CPU project anecdotally has about three times as many specification and validation engineers as design engineers and a 5:1 ratio is not unheard of

The reason silicon design has such verification to design ratio is because the cost of one bug is many, many orders of magnitude higher than software. Both in dollar cost and in schedule cost (it takes months to fab a chip, and if you messed up and need to spin a fix, it costs tens of millions of dollars, not counting any design engineering cost).

I don't think you can extrapolate these very industry-specific facts to judging LLMs.

  if an AI agent does something, you (the prompter) are responsible by default, unless you can show that your the agent itself behaved in an unexpected way and that you in no way prompted or hinted at the bad behavior, in which case the model provider is liable

On the other hand, I wonder if we'll end up with another variation of the cookie law, where every AI user or vendor just adds "don't do anything illegal" as part of their prompt to defend against that law. Thoughts?

Unlike biological viruses, AI can't replicate GPUs for free and grow.

I think that's where the future is going. It presents interesting challenges, but also some opportunities to make life better for everyone.

[1*] https://foundrs.com/have_your_ai_talk_to_my_ai.html

(*) I dislike the HN trend of starting citations at 0, I find it snobbish.

I don't know anything about the Linux scheduler, but if one core out of 96 is busy chasing after those ghost allocations, why would a network thread not use one of the other 95 cores, and instead be stuck for seconds? Unless that one core is blocking the entire kernel, but then that would be a lot more visible instantly.

Professionally, assembly is the last thing I would let an LLM generate for me. That's because if you need to write something in assembly, it's because it's critical: operating system context switch, interrupt handler, that kind of thing. You don't want to be 99% correct, you need that code to be 100% good.

I can imagine some other professional scenarios that I'm less familiar with, where you want to write a tight numeric loop of NEON64 assembly and let the LLM do it, because it's frankly a pain to do by hand, and you can test it to some degree of confidence.

But there’s nothing wrong with a car made by a robot factory. The car is useful, regardless of how it was made. AI lets some people create new products, those may be useful.

If you think a page is too vague, use a famous known writer's work as a reference.

Comments URL: https://news.ycombinator.com/item?id=48690101

Points: 1184

# Comments: 1240

High-risk – AI applications that are expected to pose significant threats to health, safety, or the fundamental rights of persons. Notably, AI systems used in health, education, recruitment, critical infrastructure management, law enforcement or justice. They are subject to quality, transparency, human oversight and safety obligations

That's a pretty common sense legislation to me.

That is not my experience mentoring 100+ startup founders. Building a prototype, the gateway to serious customer engagement, used to take months and many startups would die before finishing their first one.