Comments URL: https://news.ycombinator.com/item?id=46340123

Points: 3

# Comments: 0

online writing before 2022 is the low-background steel of the information age. now these models will all be training on their own output. what will the consequences be of this?

The classical approach was to understand how genes transcribe to mRNA, and how mRNA translates to polypeptides; how those are cleaved by the cell, and fold in 3D space; and how those 3D shapes results in actual biological function. It required real-world measurement, experiment, and modeling in silico using biophysical models. Those are all hard research efforts. And it seems like the mindset now is: we've done enough hard research, let's feed what we know into a model, hope we've chosen the right hyperparameters, and see what we get. Hidden in the weights and biases of the model will be that deeper map of the real world that we have not yet fully grasped through research.

But the AI cannot provide a 'why'. Its network of weights and biases are as unintelligible to us as the underlying scientific principles of the real world we gave up trying to understand along the way. When AI produces a result that is surprising, we still have to validate it in the real world, and work backwards through the hard research to understand why we are surprised.

If AI is just a tool for a shotgun approach to discovery, that may be fine. However, I fear it is sucking a lot of air out of the room from the classical approaches. When 'AI' produces incorrect, misleading, or underwhelming results? Well, throw more GPUs at it; more tokens; more joules; more parameters. We have blind faith it'll work itself out.

But because the AI can never provide a guarantee of correctness, it is only useful to those with the infrastructure to carry out those real-world validations on its output, so it's not really going to create a paradigm shift. It can provide only a marginal improvement at the top of the funnel for existing discovery pipelines. And because AI is very expensive and getting more so, there's a pretty hard cap on how valuable it would be to a drugmaker.

I know I'm not the only one worried about a bubble here.

Dare I ask why?

I’m loving all the feedback and wanted to address some things:

* Yes, nonce tracking is expensive. HTTP request signatures could be used instead of nonces—they’re just not fully fleshed out yet, from what I can tell. And a lot of other crypto systems we rely on in Web traffic today also assume proper nonce tracking. Fortunately you only need to track distinct nonces per established session, so you could wait to allocate storage for them until the client actually tries to set up a WebSession, and use a Bloom filter to save on storage at the risk of some false positives for nonce reuse (could be risky if you’re relying on that as tamper evidence, but you could tune your filters accordingly).

* The stuff about the ergonomics of cookies and banners and auto-log-out as they currently are is mostly incidental. I’m just trying to paint a picture of how tighter integration of the notion of a ‘session’ into the browser itself could improve the UX we currently have today.

* Many people have pointed out that this is not stateless. Indeed! This is supposed to be a more secure way of establishing a stateful connection over a stateless protocol, for which there is already a demand which we’re meeting with the least secure possible method (bearer tokens sent in plaintext over the channel). Issues with scalability need to be addressed on the implementation side, but I believe the protocol is still sound and not inherently unscalable (no more than TLS in its current form).

I am going to look for opportunities to fold these points into the document itself, and maybe rearrange some of it so that these points get covered earlier and more clearly.