I have a static server of my own, so here's my list (of all the tests I published so far): https://senko.net/vibecode-bench/

But I just vibe-coded a handy list of all the tests I did (unfortunately without the commentary I usually leave in social media posts -- I should add those at some point): https://senko.net/vibecode-bench/

After some interrogation, here's how it organized the work:

1. Design workflow (rts-game-design, 11 agents, ~13 min) ran first, produced SPEC.md + DESIGN.md:

1.1. Proposals (3 parallel agents): each designed a complete RTS from a different philosophy

1.2 Judge (1 agent): evaluated all three and synthesized one unified design, committing to specific numbers (costs, HP, map size, etc.).

1.3 Deep-dives (6 parallel agents): each wrote an implementation-ready spec for one subsystem, all consistent with the chosen design

1.4 Synthesis (1 agent): merged the design + all six subsystem specs into one conflict-free master spec

2. Code-review workflow (rts-code-review, 25 agents, ~5 min), ran after the main agent had written and tested the code:

2.1 Review (6 agents, read-only Explore type): each scrutinized one dimension and returned structured findings.

2.2. Verify (19 agents): every finding got its own skeptic agent told to try to refute it, Result: 19 flagged → 16 confirmed, 3 rejected as non-bugs.

What the main agent did in the main loop:

- Wrote all ~2,400 lines of index.html by hand from the spec.

- All browser testing/debugging via headless Chrome (I told it to use rodney by @simonw, love the tool :)

- Applied all 16 fixes from the review and re-verified them in the browser.

Not sure why it did that. Its own rationale (which is highly suspect, but the only lead I have) is that it defaults to dense style if it has to write a file in a single go. May be a kernel of truth somewhere in there.

https://bsky.app/profile/senko.net/post/3mmwnrkwboc2v

The prompt was: Create a simple but functional real time strategy (RTS) game similar to old WarCraft, StarCraft or Command & Conquer games. The player should be able to build buildings, create units, gather resources and should uncover the whole map. No AI or multiplayer needed. Use simple but nice-looking graphics. No sound. Implement everything in HTML/CSS/JS, everything in a single file (you can use 3rd-party js or css libraries/frameworks via CDN).

* if you're left-handed, your hand smudges over the ink before it dries. There are various contortions that some left-handed people do (hover the hand or wrap it around from above) - right handed ones don't need any of that.

* stroke patterns, as usually learnt in school, result in pushing away if left handed, vs drawing to, if you're right handed. This results in less ideal strokes, and if you're working with a sharp pencil/pen on a sensitive paper, this can tear the paper. If you're working with a felt-tip pen, the line width/pressure suffers as well.

That said, if you really make an effort, you can have a pretty decent handwriting if you're left handed. And if you are forced to use right hand when learning handwriting, you can still have a pretty decent handwriting.

I'm not familiar with details of chinese handwriting (what's easier/better if you're left vs right handed), wouldn't be surprise the constraints are similar.

So I guess your remark about messy handwriting is related to the strict standards for the students (which includes expectation they must write with right hand).

It's too trivial for anyone to be selling that. And I don't think there's a large market for $5-$10 barebones setup when GH is free and you can self host.

I had to authorize something with Firebase, for which I had to auth with Google, for which I had to do a MFA with my (Pixel) phone.

Usually it's "are you trying to auth" and finger-to-the-scanner, but around that time this particular way didn't work. It also didn't want to send me a text or a call to auth me.

No, I had to find an OTP code. Easy, right? Wrong. The instructions, and the docs, don't match where it was in that particular version of Android, and there were a bunch of blind alleys that were named basically the same.

It took me like 10 minutes, on stage, browsing my phone (thankfully, not casted to screen) to find the friggin' option. Thankfully the cohost was doing the presenting at that time, but it was pretty lousy.

And this is using Google's OS on a Google phone doing a Google auth flow for a Google property. And I'm a techie who's been using Android for 15+ years now. And I did the exact same dance a few weeks before that - also so roundabout I had no idea how I stumbled on the correct page.

User experience my ass.

PS. The regular "are you trying to sign in?" flow works again. No idea what happened - wasn't me.

Sooo... don't use it?

There are plenty of open source alternatives, and I'm sure someone's going to mention org-mode.

Many of these companies do build up internal know-how.

In case of big consulting companies, they have huge internal knowledge bases, decks, cases etc from previous engagements. In case of product companies with FDEs, the feedback from customers ideally trickles down to product improvements. In case of OpenAI, they can improve their models.

Whether that's actually valuable enough to turn a blind eye to the downsides of consulting, I don't know.

The trouble with dev agencies, "services companies", integration specialists and "forward deployed engineers" is that they scale lineraly with the number of people.

You can't 100x your revenue without at least 80x-ing your headcount.Oh, you might go for that once due to AI - but so can everyone else. After that, it's boring linear growth.

When I say "boring", I don't mean as "capitalism requires exponential growth" critique. I mean OpenAI valuation is not priced for that. They're priced for singularity. If the bulk of their revenue turns out to be bodyshop, that's...quite a different math.

The way to charge big with this kind of work is to do what big consultancies (MBB, IBM, etc) do: brand equity and (supposed) expertise in solving domain problems. OpenAI has ... interesting tech.

It's going to be interesting seeing if they can pull this off. If I were a betting man, my money would be on "no".

Comments URL: https://news.ycombinator.com/item?id=48095144

Points: 5

# Comments: 0

> Stop shipping distributed systems when you meant to ship a feature.

But not in the contex the author meant.

Many people don't realize that when you have a frontend, a backend (several instances, for failover/scaling), a (separate) database, maybe some object store -- you have a distributed system.

A recent article[0] touched on that, although most HN commenters[1] latched on the "go" part. But there's something to avoiding rube goldberg machines where we don't need them.

[0] https://blainsmith.com/articles/just-fucking-use-go/

[1] https://news.ycombinator.com/item?id=48062997