https://meded.ucsf.edu/willed-body-program :)

Re: favorite systems for migrations — not really; I've always just kind of not used one, or rolled my own. Desiderata:

* fully atomic (all goes through or none goes through)

* low-boilerplate

* can include execution of arbitrary application code — data-query-only only migrations feel kind of limiting.

* painless to use with multiple developers multiple of which might be writing migrations

(2) When a schema is provided, is it fully enforced? Is there a way to do migrations?

Migrations are the only remaining challenge I can think of that could screw up this tool long-term unless a good approach gets baked in early. (They're critically important + very often done poorly or not supported.) When you're dealing with a lot of data in a production app, definitely want some means of making schema changes in a safe way. Also important for devex when working on a project with multiple people — need a way to sync migrations across developers.

Stuff like scalability — not worried about that — this tool seems fundamentally possible to scale and your team is smart :) Migrations though... hope you focus on it early if you haven't yet!

Forms just inherently can have partially-finished/invalid states, and it feels wrong to try and kraal model objects into carrying intermediary/invalid data for them (and in some cases won’t work at all, eg if a single form field is parsed into structured data in the model)

It's easy to say just "use containers" or "use VMs" — but are there pragmatic workflows for doing these things that don't suffer from too many performance problems or general pain/inconvenience?

Are containers the way to go, or VMs? Which virtualization software? Is it best to use one isolated environment per project no matter how small, or for convenience's sake have a grab-bag VM that contains many projects all of which are low value?

Theorycrafting is welcome, but am particularly interested in hearing from anyone who has made this work well in practice.

Comments URL: https://news.ycombinator.com/item?id=41259900

Points: 27

# Comments: 21

Wait, what? Where did they get human fat from…?

Comments URL: https://news.ycombinator.com/item?id=39064645

Points: 3

# Comments: 0

Naively, it seems reasonable to believe that if you adjust all the weights of a neural net towards the behavior you want via SFT and RLHF, that it would compete with/mute/obscure undesired behavior like a back door. But it seems not to be so… Indeed the cute mask does not cover the entire shoggoth— it may still have tentacles (https://images.app.goo.gl/YW9g3BvwGqGwYTgd6)

AutoGPT itself is pretty weak, but it’s possible to write wrapper code that leads to stronger agency. Also, agents formed this way with GPT4 are way stronger than with GPT3.5… so expect this trend to continue.

This is often said, but it isn't so.

The task of predicting the next token in human speech really well requires immense intelligence — potentially far more intelligence than possessed by the original speaker! Imagine yourself engaging in the task of listening to someone who isn't that smart speak and then trying to figure out what they'll say next — in doing so, you might make all sorts of extrapolations about the person, their motivations, their manner, their dialect, etc — calling on all sorts of internal models that you've built up about people over time. This is what models are being trained to do when we train them on predicting tokens.

There are concrete examples of models inventing new ways of thinking that are not described in their training set. For example, when training a transformer from scratch to perform addition mod P (and having no training data other than examples of addition mod P), the transformer was able to discover the use of discrete fourier transforms and trigonometric identities [1]. As we can see, neural nets can build all sorts of internal mental models that no one explained to them beforehand. These internal mental models can then be elicited and used for other purposes by e.g. fine-tuning.

I think a good mental model for transformers/neural nets is that they're automatic scientists. They figure out ways of modeling things in order to predict the output from the input — which is what scientists do! As part of this, they can de-facto discover new theories, and come to rely on the theories that prove useful in their prediction task.

Also, not all tokens in the training set are from human speech, so models are being trained to model all manner of data-generating processes.

[1] https://arxiv.org/pdf/2301.05217.pdf

Do you think https://github.com/Significant-Gravitas/Auto-GPT et al will become more performant as models improve?

```

You are a super intelligent honest question-answering system.

Q: What's 2+2?

A: 4

Q: What color is the sky?

A:

```

Caro’s works have spoiled me — other biographies feel out-of-focus and surface-level and speculative.

(And all this despite the fact that Caro isn’t even really focusing directly on his subjects, because actually his books are about power and its dynamics.)