What _content_ will bots/LLMs actually have access to in... 3 years? PubMed, Wikipedia, and various company's sales decks and documentation? What incentives exist in the future for people to actually produce content? The existing framework is being disrupted and a lot of the creator/publisher economy's revenue is, seemingly, anticipated to flow into Anthropic/OpenAI.

I like the premise of Cloudflare's "pay-per-crawl" as a bridge to preserve publisher incentives, but that only really works if you assume repeat access and not storage? I just don't really understand what the next paradigm of attention-revenue will look like. There are two edges: large publishing deals, ultimately symbolizing collective bargaining against bot traffic, and trusted curation: is this best represented by substack?

I remember when the online community was up in arms around Google's Accelerated Mobile Pages product and the degree to which it dis-intermediated content creators and hosts. At least in that plan, publishers still retained the "click." AMP underwent some major changes due to regulatory (anti-trust) pressure and publisher feedback (if memory serves), specifically around the promotion of publishers that _utilized_ AMP.

Many of us lament the degree to which content is valuable to producers simply because it draws attention (clickbait, outrage slop). Is metered-rate access to content actually going to disentangle Attention from the Producer's incentives? Will content be monetized on the degree to which it is useful? Does this more cleanly separate "entertainment" from "information?" What outrageous politics/economics will be involved in a provider becoming "trusted source" for LLMs?

The social impact of the transition I'm projecting here depends a _lot_ on whether chat services / LLM providers are able to _resist_ the revenue (or performance metrics) that is/are available to companies that experiment their way into being outrage machines and echo chambers. LLMs don't inherently provide any solution to this problem and, in fact, _facilitate_ sycophancy in mind-numbing and society-altering new ways.

From the perspective of... "information health?" this could theoretically be an amazing step forward -- information is as useful as it is accurate. Realistically, we are just shifting the attention incentives onto the model/system provider who have a far greater capacity to "tune" their "algorithm" towards delighting their customers by manipulating prompts. See: sycophancy. Attention optimization per user. How will answer-engine optimization (AEO) end up re-articulating the lessons of SEO?

To try to clean this up, there are two massive "incentive geometry" transitions at work: 1. monetization of content production for bot-driven access 2. incentive structure of the consumption interface

Production monetization lives and dies by the _perishability_ of the published information.

The consumption interface relocates the attention pathology upstream and potentially weaponizes it for an audience of one, entirely within the closed walls of a model provider. What can possibly counterbalance the raw economic forces at work other than civic effort (regulation, collective action, cultural norms)? Is it meant to be competition?

Can "information quality" be a market signal when the buyer (model providers or chat interface providers) have entirely different incentives than the end user? Didn't we already answer this conclusively with SEO?

I'm curious what other HN readers think is likely, given the current social, political, and educational climate.

In the framing of using LLMs as legal tutors, with the implication of lowering the cost of legal training, this seems like a socially-positive outcome. Furthermore, it feels kind of intuitive to me that any contemporary system operating with an LLM and access to legal reference material will be prepared to answer _student-originated questions_ comprehensively and with breadcrumbs or direct references to educational/source materials, as seems to have been found in the study.

The authors explicitly and intentionally emphasize that many legal questions require contextualization, as opposed to some discrete calculated answer. The result of the study implies that the LLM-based systems were capable of using what many of us here understand to be the "stochastic best-fit algorithmic generation" of a contemporary language model to adequately contextualize a student's question, providing insight into the trade-offs or complications implicit in the question, while then, critically, _meeting the professional standards of legal educators in explaining that complexity to a student_.

Realistically, I would hope this provides some confidence to readers of HN that they can actually ask a legal question to an LLM and expect the response will explain the complexity of the law in relation to the question. This is great news, and is likely the minimal pre-work any of us should do before actually consulting a lawyer, if time permits.

On the other hand, I do _not_ think that this study provides any indication that an LLM is prepared to actually provide direct legal counsel. Possibly in the same way that a legal textbook does not replace legal counsel, or perhaps more accurately, the same way that stumbling upon a legal case study for approximately the same situation you're in doesn't guarantee you'll have the same result.

Contextualized to "web-apps," as you have; navigating a list maybe requires an interface. It would be fairly tedious to differentiate between, for example, the 30 pairs of pants your computer has shown you after you asked "help me buy some pants" without using a UI (ok maybe eye-tracking?).

This project would be the developer tool used to produce interactive tools for end users.

More practically, it just redefines the developer's position; the developer and end-user are both "users". So the developer doesn't need to think AND the user doesn't need to think.

Kind of like saving a game before taking on a boss. If things go haywire, just reload. Or maybe like cooking? If something went catastrophically wrong, just throw it out and start from the beginning (with the same tools!)

And I think the only way to even halfway mitigate the vulnerability concern is to identify that this hypothetical system can only serve a single user. Exactly 1 intent. Totally partitioned/sharded/isolated.

Abstractly, who cares what format the information is shared in? If it is complete, the rigidity of the schema *could* be irrelevant (in a future paradigm). Determinism is extremely helpful (and maybe vitally necessary) but, as I think this intends to demonstrate, *could* just be articulated as a form of optimization.

Fluid interpretation of API results would already be useful but is impossibly problematic. How many of us already spend meaningful amounts of time "cleaning" data?

[1]: https://github.com/deepseek-ai/DeepSeek-OCR/blob/main/DeepSe...

The hard part (coming from this direction) is enshrining the translation of specific user intentions into deterministic outputs, as others here have already mentioned. The hard part when coming from the other direction (traditional web apps) is responding fluidly/flexibly, or resolving the variance in each user's ability to express their intent.

Stability/consistency could be introduced through traditional mechanisms: Encoded instructions systematically evaluated, or, via the LLMs language interface, intent-focusing mechanisms: through increasing the prompt length / hydrating the user request with additional context/intent: "use this UI, don't drop the db."

From where I'm sitting, LLMs provide a now modality for evaluating intent. How we act on that intent can be totally fluid, totally rigid, or, perhaps obviously, somewhere in-between.

Very provocative to see this near-maximum example of non-deterministic fluid intent interpretation>execution. Thanks, I hate how much I love it!