This can be determined by offline benchmarks if you build a system that takes small sequences of actions, and requires live ab test for long sequences of actions.

The more your human ops team work from documented standard operating procedure, rather than tacit knowledge, the less you need Evals except to capture edge cases

I agree with that Evals are a scarce commodity rn. A business needs to define what good looks like. This is a laborious, and sometimes politically controversial, process.

Given good Evals, frontier llms are a magical tool that can automate tasks and do them more accurately than human ops teams.

Exactly which llm to use depends on the mixture of speed, cost and quality of the output.

Jev makes a claim to expand some regions of the Pareto frontier. I look forward to testing if this is true.

There are many areas of work we can’t automate rn. We cannot create good Evals either because time horizons are too long, or it’s too difficult to create good Evals.

That doesn’t mean there’s anything wrong with building good ai engineering systems in areas where it works magically.

Would this ever be viewed as an abuse of dominant market position, aiming to disadvantage potential entrants.

E.g. map -> THINK -> USER -> TOOL

If they learn something specific in the chat finetuning stage, this might show LLM its user input text not these tag references.

Comments URL: https://news.ycombinator.com/item?id=47838626

Points: 1

# Comments: 0

Comments URL: https://news.ycombinator.com/item?id=47206009

Points: 863

# Comments: 359

This is a really nice open source coding agent implementation. The use of async is interesting.

Comments URL: https://news.ycombinator.com/item?id=46476716

Points: 1

# Comments: 0

Comments URL: https://news.ycombinator.com/item?id=46359761

Points: 1

# Comments: 0

Comments URL: https://news.ycombinator.com/item?id=46262574

Points: 2

# Comments: 0

It is believed dense models cram many features into shared weights, making circuits hard to interpret.

Sparsity reduces that pressure by giving features more isolated space, so individual neurons are more likely to represent a single, interpretable concept.

Comments URL: https://news.ycombinator.com/item?id=45802497

Points: 2

# Comments: 0

It requires no local gpus, just creating a json and posting to OpenAI

https://platform.openai.com/docs/guides/model-optimization

https://noyb.eu/en/noyb-files-complaint-against-eu-commissio...

https://eur-lex.europa.eu/EN/legal-content/summary/transpare...

Comments URL: https://news.ycombinator.com/item?id=45316053

Points: 2

# Comments: 0

This gives the probability that two randomly chosen customers belong to the same firm.

In one micro models of oligopoly, Cournot competition, it lines up directly with the markup firms can sustain.

Outside of theory, it’s an intuitive way to average together the market power of all firms, with increases in market share for bigger players being weighted more heavily.

https://chatgpt.com/share/6897a21b-25c0-8011-a10a-85850870da...

Pretty interesting - some contamination, some better answers, and it failed to write a sentence with all 5-letter-words. I’d have expected it to pass this one!

Simple example: “Every night, dreams swirl swiftly.

Comments URL: https://news.ycombinator.com/item?id=44705070

Points: 6

# Comments: 1

Comments URL: https://news.ycombinator.com/item?id=44586786

Points: 4

# Comments: 0

Comments URL: https://news.ycombinator.com/item?id=44404983

Points: 3

# Comments: 1