Good code makes it easier for the business to move fast and stay ahead of the competition while reducing expenses for doing so.

A recurrent neural network implementing binary addition with carry could do this, and one can derive the correct weights with pen and paper without too much effort.

Whether gradient descent will find them too is another matter entirely

How do you do this? Do you follow traditional testing practices or do you have novel strategies like agents with separate responsibilities?

Sharing code enables others to validate the method on a different dataset.

Even before LLMs came around there were lots of methods that looked good on paper but turned out not to work outside of accepted benchmarks

Meanwhile, running opencode in a podman container seems to stop this particular, err, feature.

Indeed, competition is undesirable for all participants involved: everybody wants to win and exploit the rest for their own gain. Note that this is the only way to make competition work and result in its temporary benefits (if nobody wants to win, nobody will compete).

So there must be a system to keep the competition going and preventing the rise of a definitive and exploitative winner, and the existence of this system has to be accepted by the competitors. But why would serious competitors accept a system that prevents them from winning?

As the provider of a public place, X ought to take certain measures to ensure public safety on its premises. Of course, deciding what is and in not tolerable is the crux of the issue, and is far from trivial.

A look at how European governments negotiate with pharma companies helps explain why Americans pay more for prescription drugs.

https://news.stanford.edu/stories/2024/11/what-other-countri...

https://en.wikipedia.org/wiki/Pharmacovigilance#Adverse_even...

This book is very strong on the fundamentals, while the R code is minimal and easy to follow.

> how important is computing the exact gradient using calculus

Normally the gradient is computed with a small "minibatch" of examples, meaning that on average over many steps the true gradient is followed, but each step individually never moves exacty along the true gradient. This noisy walk is actually quite beneficial for the final performance of the network https://arxiv.org/abs/2006.15081 , https://arxiv.org/abs/1609.04836 so much so that people started wondering what is the best way to "corrupt" this approximate gradient even more to improve performance https://arxiv.org/abs/2202.02831 (and many other works relating to SGD noise)

> vs just knowing the general direction to step

I can't find relevant papers now, but I seem to recall that the Hessian eigenvalues of the loss function decay rather quickly, which means that taking a step in most directions will not change the loss very much. That is to say, you have to know which direction to go quite precisely for an SGD-like method to work. People have been trying to visualize the loss and trajectory taken during optimization https://arxiv.org/pdf/1712.09913 , https://losslandscape.com/

Scientists and academics demand an entirely different level of rigor compared to customers of LLM providers.