- They might be dynamically adjusting these at inference time [1]. For example, start with a low temperature and generate samples with increasingly high temperatures until one of them passes some quality gate.

- They don't want you to fine-tune on high temperature completions (rejection fine-tuning). You could call this "rejection fine-tuning rejection".

[1] https://rlhfbook.com/c/09-rejection-sampling#related-best-of...

Comments URL: https://news.ycombinator.com/item?id=49000641

Points: 2

# Comments: 0

On the other hand, $10 for 1M tokens still seems really high? It's not too hard to blow through that in an hour or two.

I write technical blog posts with visualizations and live demos. That usually means embedding a bit of custom javascript in the page for the demo. Or shipping custom wasm to enable extreme semantic model compression.

I do this by pushing content from my machine to github pages which is wired up to my subdomain.

If github pages stops being a good, free option for this, I will find another. Not sure I would call this "hardcore" really.

Comments URL: https://news.ycombinator.com/item?id=48727661

Points: 4

# Comments: 0

Comments URL: https://news.ycombinator.com/item?id=48642299

Points: 5

# Comments: 0

A word's "difficulty" would be some function of how rare it is. Once you have a reasonable estimate of the user's "skill" you can infer that a user won't know more difficult words. The benefit of this is you're not spending time asking the user about words they probably know.

Of course it's possible at an individual level, difficulty does not monotonically increase as a function of how rare the word is. A person might be very familiar with a domain-specific subset of English. But the "stratified sampling" approach will also have this problem.

There is a similar problem in chess, where players have ratings which really only change on one dimension. So there can theoretically be a mismatch when puzzles are also scored on a single axis, since a "harder" puzzle that contains a motif a player is familiar with will actually be easier for the player.

There was a decent amount of work involved in getting the download size reasonable since we need to store all valid moves in a position. There are puzzles with over 40 million valid move sequences, so I had to aggressively prune and compress the move trees.

[1] https://apps.apple.com/us/app/one-million-checkmates/id67625...

Comments URL: https://news.ycombinator.com/item?id=48189178

Points: 97

# Comments: 25

I'll second this. Much better to set up a second machine you can ssh/tailscale into. If a training run takes down your training machine, you don't want it to also take down your home entertainment server.

Comments URL: https://news.ycombinator.com/item?id=47212819

Points: 2

# Comments: 0

So in practice, a drop in win probability greater than 14% is considered a blunder on Lichess.

For reference, lichess uses the following function to map centipawn advantage to the probability bar, derived from observed outcomes: https://github.com/lichess-org/lila/pull/11148

From an ML perspective, this is basically logistic regression with a single feature. However, once we leave the realm of theoretical centipawn value and begin to optimize predictive power, we could imagine adding in other things like the players' ELOs or time remaining per player, etc.

I think there are some interesting theoretical differences between predicted win probability derived from Stockfish CP and actual outcomes. As in, you could even imagine predicting positions where certain players struggle and steering them towards those positions. [0]

[0] https://www.youtube.com/watch?v=KgOC1D8wkyE

Comments URL: https://news.ycombinator.com/item?id=47105038

Points: 47

# Comments: 4

That said, I probably wouldn't use this unless mine was one of the specific use cases supported[0]. I have no idea how hard it would be to add a new model supporting arbitrary inputs and outputs.

For running inference cross-device I have used Onnx, which is low-level enough to support whatever weights I need. For a good number of tasks you can also use transformers.js which wraps onnx and handles things like decoding (unless you really enjoy implementing beam search on your own). I believe an equivalent link to the above would be [1] which is just much more comprehensive.

[0] https://ai.google.dev/edge/mediapipe/solutions/guide

[1] https://github.com/huggingface/transformers.js-examples

That goes double for visuals, which is another reason I use a custom static site. Can't run JS on Medium. I even built out a client-side search and learned a good amount from that too.

Comments URL: https://news.ycombinator.com/item?id=42908500

Points: 2

# Comments: 0