Comments URL: https://news.ycombinator.com/item?id=48556561

Points: 3

# Comments: 2

We got tired of browser frameworks restricting the LLM, so we removed the framework and gave the LLM maximum freedom to do whatever it's trained on. We gave the harness the ability to self correct and add new tools if the LLM wants (is pre-trained on) that.

Our Browser Use library is tens of thousands of lines of deterministic heuristics wrapping Chrome (CDP websocket). Element extractors, click helpers, target managemenet (SUPER painful), watchdogs (crash handling, file downloads, alerts), cross origin iframes (if you want to click on an element you have to switch the target first, very anoying), etc.

Watchdogs specifically are extremely painful but required. If Chrome triggers for example a native file popup the agent is just completely stuck. So the two solutions are to: 1. code those heuristics and edge cases away 1 by 1 and prevent them 2. give LLM a tool to handle the edge case

As you can imagine - there are crazy amounts of heuristics like this so you eventually end up with A LOT of tools if you try to go for #2. So you have to make compromises and just code those heuristics away.

BUT if the LLM just "knows" CDP well enough to switch the targets when it encounters a cross origin iframe, dismiss the alert when it appears, write its own click helpers, or upload function, you suddenly don't have to worry about any of those edge cases.

Turns out LLMs know CDP pretty well these days. So we bitter pilled the harness. The concepts that should survive are: - something that holds and keeps CDP websocket alive (deamon) - extremely basic tools (helpers.py) - skill.md that explains how to use it

The new paradigm? SKILL.md + a few python helpers that need to have the ability to change on the fly.

One cool example: We forgot to implement upload_file function. Then mid-task the agent wants to upload a file so it grepped helpers.py, saw nothing, wrote the function itself using raw DOM.setFileInputFiles (which we only noticed that later in a git diff). This was a relly magical moment of how powerful LLMs have become.

Compared to other approaches (Playwright MCP, browser use CLI, agent-browser, chrome devtools MCP): all of them wrap Chrome in a set of predefined functions for the LLM. The worst failure mode is silent. The LLM's click() returns fine so the LLM thinks it clicked, but on this particular site nothing actually happened. It moves on with a broken model of the world. Browser Harness gives the LLM maximum freedom and perfect context for HOW the tools actually work.

Here are a few crazy examples of what browser harness can do: - plays stockfish https://x.com/shawn_pana/status/2046457374467379347 - sets a world record in tetris https://x.com/shawn_pana/status/2047120626994012442 - figures out how to draw a heart with js https://x.com/mamagnus00/status/2046486159992480198?s=20

You can super easily install it by telling claude code: `Set up https://github.com/browser-use/browser-harness for me.`

Repo: https://github.com/browser-use/browser-harness

What would you call this new paradigm? A dialect?

Comments URL: https://news.ycombinator.com/item?id=47890841

Points: 134

# Comments: 66

At Browser Use we spend a lot of time thinking about the bitter lesson of agentic frameworks. A few days we did an experiment that combines replacing Playwright with CDP and giving the LLM complete freedom (building with autoresearch from ground up) to do whatever it knows from pretraining.

I think this is the simplest way to give an AI control of a real browser: raw CDP, and let the agent write its own tools. ~600 lines total.

Example: I forgot to implement upload_file(). Mid-task, the agent noticed, wrote the function, and uploaded the file. I found out when I read the git diff.

I haven't found a task that doesn't work yet - but extremely happy to be proven wrong. Give it a shot. Open to criticism on CDP, agent frameworks, browser use, or whatever.

Comments URL: https://news.ycombinator.com/item?id=47829234

Points: 3

# Comments: 1

Comments URL: https://news.ycombinator.com/item?id=47669995

Points: 3

# Comments: 0

Comments URL: https://news.ycombinator.com/item?id=47374619

Points: 1

# Comments: 0

Comments URL: https://news.ycombinator.com/item?id=47181316

Points: 79

# Comments: 17

Comments URL: https://news.ycombinator.com/item?id=47123845

Points: 1

# Comments: 0

At that point it might be just easier to convince the model to write JS directly

Comments URL: https://news.ycombinator.com/item?id=46654581

Points: 2

# Comments: 0