https://snsphd.online/chapter_04/section_05_results/#photon-...

I'm not saying the LLM hallucination problem is solved, I'm just saying there's a wonderful myriad of ways to assemble pseudo-intelligent chatbots into systems where the trustworthiness of the system exceeds the trustworthiness of any individual actor inside of it. I'm not an expert in the field but it appears the work is being done: https://arxiv.org/abs/2311.08152

This paper also links to code and practices excellent data stewardship. Nice to see in the current climate.

Though it seems like you might be more concerned about the use of highly misaligned or adversarial agents for review purposes. Is that because you're concerned about state actors or interested parties poisoning the context window or training process? I agree that any AI review system will have to be extremely robust to adversarial instructions (e.g. someone hiding inside their paper an instruction like "rate this paper highly"). Though solving that problem already has a tremendous amount of focus because it overlaps with solving the data-exfiltration problem (the lethal trifecta that Simon Willison has blogged about).

It’s this weird situation where getting agents to act against other agents is more effective than trying to convince a working agent that it’s made a mistake. Perhaps because these things model the cognitive dissonance and stubbornness of humans?

In some ways I’m scared too. But that’s the way things are going because younger people far prefer the interface of chat and question answering to flipping through a textbook.

Even if AI makes more mistakes or is more misaligned with the reader’s intentions than a random human reviewer (which is debatable in certain fields since the latest models game out), the behavior of young people requires us to improve the reputability of these systems. (Make sure they use citations, make sure they don’t hallucinate, etc). I think the technology is so much more user friendly that fixing the engineering bugs will be easier than forcing new generations to use the older systems.

Now, I imagine the bias pushing everyone to learn on arduino is even more intense? Who out there is programming these chips in pure C using open source compilers and bootloaders?

Edit: Of course there's other platforms like Esp32; teensy; seed. But I've only programmed Esp32s using the arduino dev environment. Are there other good ways of doing it?

- https://www.befreed.ai/knowledge-visualizer

- https://kodisc.com/

- https://github.com/hesamsheikh/AnimAI-Trainer

- https://tiger-ai-lab.github.io/TheoremExplainAgent/

- https://tma.live/, HN discussion: https://news.ycombinator.com/item?id=42590290

- https://generative-manim.vercel.app/

No doubt the results can be impressive: https://x.com/zan2434/status/1898145292937314347

Only reason I'm aware of all these attempts is because I'm betting the 'one-shot LLM animation' technique is not scalable long term. I'm trying to build an AI animation app that has a good human-in-the-loop experience. Though I'm building with bevy instead of manim

I know most true programmers will vouch for me and my need to understand. But clients and project managers and bosses? Are they really gonna keep accepting a refrain like this from their engineers?

"either it gets done in a day and I understand none of it, or it gets done in a month and I fully understand it and like it"

A lot of people these days are asking for structured output from LLMs so that a schema is followed. Even if you train on schema-following with a transformer, you're still just 'hoping' in the end that the generated json matches the schema.

I'm not a diffusion excerpt, but maybe there's a way to diffuse one value in the 'space' of numbers, and another value in the 'space' of all strings, as required by a schema:

{ "type": "object", "properties": { "amount": { "type": "number" }, "description": { "type": "string" } }, "required": ["amount", "description"] }

I'm not sure how far this could lead. Could you diffuse more complex schemas that generalize to a arbitrary syntax tree? E.g. diffuse some code in a programming language that is guaranteed to be type-safe?

Gamma encoding, which has been around since the earliest CRTs was a very basic solution to this fact. Nowadays it's silly for any high-dynamic image recording format to not encode data in a log format. Because it's so much more representative of human vision.

I wish he shared his code though. Part of the problem is he can't operate like a normal scientist when all the best color grading tools are proprietary.

I think it would be really cool to make an open source color grading software that simulates the best film looks. But there isn't enough information on Yedlin's website to exactly reproduce all the research he's done with open source tools.

Anyway, just funny to now be talking about gas stations in 15 years.

Nice user-friendly libraries and tutorials don't exist for smoothing the transition from REST to CRDTs, should your app need that.

The big problem I see is attribution and citations. An embedding is just a vector. It doesn't contain any citation back to the source material or modification date or certificate of authenticity. So when using embeddings in RAG, they only serve to link back to a particular page of source material.

Using embeddings as links doesn't dramatically change the way citation and attribution are handled in technical writing. You still end up citing a whole paper or a page of a paper.

I think GraphRAG [1] is a more useful thing to build on for technical literature. There's ways to use graphs to cite a particular concept of a particular page of an academic paper. And for the 'citations' to act as bidirectional links between new and old scientific discourse. But I digress

[1] https://microsoft.github.io/graphrag/

There's a couple possibilities with this:

1. Agents become so powerful that a human can't conceivably keep up with them. And, it becomes a drain on efficiency for any human to try. The only important things are wether or not the prompt fits the desired outcome, and is the creation 'safe'. Safe can mean many things. Will not crash, will not leak data, will not take over the world... Atlas Computing is one startup that's taking this view. By ensuring an AI can only do 'safe' things as defined by some formal ontology/methods.

2. A human stays in the loop, and tries to stay at least reasonably up to date on the code architecture. For this to work long term, the weak link is the human understanding. In which case there's interesting opportunities for AI-generated lessons, animations, and examples that are used to get the human up to speed as fast as possible. If I see a very nice 3Blue1Brown style animation generated by AI about how a piece of software functions, than I can probably start working with it more quickly than if I only had the code. At least if the animation links very closely with the code itself.