I hope that we will soon see deterministic inference offered by frontier labs (presumably at a premium rate). In the meantime, you can get deterministic behavior from local models using the right vLLM flags. I've described some of the cool applications of determinism here: https://lukechampine.com/repligraphs

True, I expect models to be pretty correlated with each other as well. But you can at least attempt to quantify that correlation in a rigorous way, by running experiments (e.g. by pointing each model at a big project and comparing the sets of bugs they each find).

And yeah, the "double-edged sword" aspect of determinism is definitely the biggest bullet that you have to bite. For me, it's better than the alternative; without determinism, such jailbreaking attacks are still possible, just harder to detect. But I certainly would not want people to go around thinking "it's deterministic, therefore it's safe."

Naturally, I hope that OpenAI and Anthropic will one day offer deterministic inference; see the "Weights" page for what this might look like.

Comments URL: https://news.ycombinator.com/item?id=49819932

Points: 3

# Comments: 2

I love programming. It's in my blood: my father and grandfather were programmers too. I have written everything from SIMD assembly to Hoon, and implemented several languages of my own. Believe me, I am intimately familiar with the phenomenon you are describing.

It's true, the devil is in the details. And I will grudgingly concede that, at present, humans are better at exorcising demons than AI. But I see no reason to believe that this will remain true. The gap is narrowing rapidly, and even today there are types of demons that AI can dispatch much more quickly and effectively than you or I can. The fact that vibe coding is possible at all (and that people are willing to pay for vibe-coded apps) is proof that an informal English prompt is sufficient to specify software to an acceptable degree. Not acceptable to everyone, naturally, but at least to the creator and the users.

I am not exactly happy about this. It is bittersweet. Much of my identity is bound up in being a programmer. The devil is in the details; but joy and whimsy and great beauty are in the details as well. For a glorious few decades, one could be an artist under the guise of producing economic value. Now, the economic aspect of producing software is being siphoned off, to be done by machines, leaving only the art. I think we will suffer for that, somewhat. But it is a small price to pay.

English is not a programming language. Yet English is sufficient to communicate requirements to the degree that we actually care about. A programmer's job is to translate English into lower-level machine language. Necessary to this process is "filling in the gaps" -- that is, extrapolating the expressed intent to cover all the little details that were left unspecified. This system works because humans are at least minimally competent at predicting the preferences of other humans. If your prediction turns out to be wrong, you get feedback and iterate.

Well, guess what. LLMs are also competent at predicting the preferences of humans. LLMs can "fill in the gaps" like no one's business. LLMs can iterate on requirements like no one's business.

Product managers do not speak to programmers in a language that encodes exact requirements, and yet working software somehow gets shipped anyway. LLMs do not need exact requirements either.

So when you write a function like:

  func hypot(x, y):
    return sqrt(x*x + y*y)

   "This is the area where Go genuinely shines, and it’s worth being precise about why"
   "the lack of GC pauses is a genuine selling point"
   "Humans are genuinely bad at reasoning about memory"
   "There are cases where the borrow checker is genuinely too strict"

> After a disorienting visit from the FBI in May of 1990, I wrote a rant called Crime and Puzzlement, which led to my establishing with Mitch Kapor (who had previously founded Lotus Development Company) an organization called the Electronic Frontier Foundation.

> Now, after almost two years of operation...