Hacker News: mlukewizard

New comment by mlukewizard in "DishwasherBench – Can Frontier Models Stack a Dishwasher?"

mlukewizard — Tue, 13 Oct 2026 15:12:02 +0000

ahahhaa

New comment by mlukewizard in "DishwasherBench – Can Frontier Models Stack a Dishwasher?"

mlukewizard — Tue, 13 Oct 2026 15:10:49 +0000

yea super interesting q

my feeling is that stacking a dishwasher involves enough image processing and computer-use (or more realistically robotics) that it's a frontier model job rather than a 'write a program to do it' job

but keen to hear if you disagree

New comment by mlukewizard in "DishwasherBench – Can Frontier Models Stack a Dishwasher?"

mlukewizard — Tue, 13 Oct 2026 14:53:24 +0000

I would totally do this, is this wrong? It still gets sprayed from the bottom no?

New comment by mlukewizard in "DishwasherBench – Can Frontier Models Stack a Dishwasher?"

mlukewizard — Tue, 13 Oct 2026 14:19:38 +0000

Mmmm yea interesting, thanks for the feedback will push an update :)

New comment by mlukewizard in "DishwasherBench – Can Frontier Models Stack a Dishwasher?"

mlukewizard — Tue, 13 Oct 2026 14:14:40 +0000

100%, users check before kicking off a run whether the drawers close and the spinner spins

New comment by mlukewizard in "DishwasherBench – Can Frontier Models Stack a Dishwasher?"

mlukewizard — Tue, 13 Oct 2026 10:51:26 +0000

Question for both: do you think that is also the case with a real-world dishwasher (it's mostly about getting stuff to fit), or do you think it's an issue with the current sim?

The sim needs work, for sure

New comment by mlukewizard in "DishwasherBench – Can Frontier Models Stack a Dishwasher?"

mlukewizard — Tue, 13 Oct 2026 09:59:28 +0000

Totally the opposite, there's a detailed dishwasher simulation under the hood: ray tracing, splash simulation, soak, heat, enzyme action

I calibrated it against my actual dishwasher by doing some test runs with baked egg

New comment by mlukewizard in "DishwasherBench – Can Frontier Models Stack a Dishwasher?"

mlukewizard — Tue, 13 Oct 2026 09:21:50 +0000

yeaaa perhaps this is the future

New comment by mlukewizard in "DishwasherBench – Can Frontier Models Stack a Dishwasher?"

mlukewizard — Tue, 13 Oct 2026 09:19:32 +0000

sure the code isn't hand-written and it's a work in progress, but it's designed to be modelled and calibrated around my own dishwasher so it's verifiably non-slop

New comment by mlukewizard in "DishwasherBench – Can Frontier Models Stack a Dishwasher?"

mlukewizard — Tue, 13 Oct 2026 08:24:06 +0000

Inspired by Joma Tech's new YouTube video

Stacked loads are run against a dishwasher simulation calibrated to my own Bosch dishwasher

Playable by both humans and AI

Currently a WIP, simulations need refining before running a full benchmark, but keen to hear some feedback!

DishwasherBench – Can Frontier Models Stack a Dishwasher?

mlukewizard — Tue, 13 Oct 2026 08:24:06 +0000

Article URL: https://github.com/composo-ai/DishwasherBench

Comments URL: https://news.ycombinator.com/item?id=50068437

Points: 89

# Comments: 54

Composo open-sources its LLM-as-Judge technique (83.6% on RewardBench 2)

mlukewizard — Thu, 02 Apr 2026 16:05:52 +0000

Article URL: https://github.com/composo-ai/llm-judge-criteria-ensembling

Comments URL: https://news.ycombinator.com/item?id=47616304

Points: 5

# Comments: 0

New comment by mlukewizard in "Ask HN: Who is hiring? (September 2024)"

mlukewizard — Fri, 06 Sep 2024 13:58:54 +0000

Composo | Founding Software Engineer | London, UK ONSITE |

https://composo.ai

Composo is on a mission to ensure that AI transforms our world effectively, rapidly, and safely. Our platform squashes the legal and reputational risk associated with shipping AI apps and improves the helpfulness of our customer’s AI agents.

We believe subject matter experts are critical to evaluating and improving AI applications. Our platform equips them a powerful evaluation toolkit in a no-code interface, while giving engineers the development freedom they need to ship their best code.

We’re looking for generalist full stack software engineers, building up to a team of 5 over the coming months. Stack is Python FastAPI on Postgres with React Typescript frontend.

Full listing here: https://composo.notion.site/Careers-at-Composo-decbf0cd01af4...

Send an email to luke[at]composo[dot]ai with your CV and a note about why Composo excites you. I will reply to all applications sent in September that clearly aren’t from bots.