Comments URL: https://news.ycombinator.com/item?id=47865741

Points: 1

# Comments: 0

My idea was that, if I see that your project is growing 10k LOC per week and you're the only developer working on it, it's most likely vibe-coded.

I analyzed some open-source projects, but unfortunately it turns out not to be so clear cut. It's relatively easy to estimate the growth rate of a project, but figuring out how much time developers worked on it is very error prone, which results in both false positives and false negatives.

I wrote a post about it (https://pscanf.com/s/352/) if you're interested in the details.

Comments URL: https://news.ycombinator.com/item?id=47833379

Points: 1

# Comments: 0

The takeaway from my experiment is that you can really tell a lot by how / when / what people commit, but conclusions are very hard to generalize.

For example, I've also stumbled upon the "merge vs squash" issue, where squashes compress and mostly hide big chunks of history, so drawing conclusions from a squashed commit is basically just wild guessing.

(The author of course has also flagged this. But I just wanted to add my voice: yeah, careful to generalize.)

¹ Nothing is ever finished.

Ironically, that comment was added three months after I posted the article, when it was nowhere near the front page anymore, in a clearly automated and AI-driven review.

Still salty about it.

I tried to implement something like this for the search functionality, but ended up going with "old school" lexical search instead.

Mostly because, in my experimentation, vector search didn't perform significantly better, and in some case it performed worse. All the while being much more expensive on several fronts: indexing time (which also requires either an API or a ~big local model), storage, search time, and implementation complexity.

And Superego's agent actually does quite well with the lexical search tool. The model usually tries a few different queries in parallel, which approximates a bit semantic search.

Correct.

> Also, how do you convert your docs / information into JSON documents?

Right now you have to add it yourself to the database. The idea is that you use Superego as the software in which you record your expenses / income / whatever, so the data is naturally there already.

But I'm also working on an "import from csv/json/etc" feature, where you drop in a file and the AI maps the info it contains to the collections you have in the database.

In my app Superego (https://github.com/superegodev/superego, shameless plug) I use structured JSON documents precisely for this reason, because with a well-defined schema the LLM can write TypeScript functions that must compile. This doesn't guarantee correctness, of course, but it actually goes a long way.

But doing my taxes was a use case I hadn't considered, and it's actually pretty neat! I'll be trying it myself next month (though I'm not looking forward to it).

I wish that more apps would instead move the logic to the client and use files on file syncing services as databases. Taking tasks as an example, if a task board were just a file, I could share it with you on Dropbox / Drive / whatever we both use, and we wouldn't need a dedicated backend at all.

The approach has limitations (conflict resolution, authorization, and latency are the big ones), but it is feasible and actually completely fine for lots of apps.

¹ Glossing over the "what they're getting in return" part. ² https://www.warpbuild.com/

If you're referring to the sandboxing / isolation of each app, I agree. Plus, the user can change the app quite easily, so if when they spot a bug, they can tell the agent to fix it (and cross their fingers!).

> ensure that a human developer understands the code before committing

Just to clarify: for Superego's app there's no human developer oversight, though. At least for the ones the user self-creates. Obviously the user will check that the app they just made works, but they might not spot subtle bugs. I employ some strategies to _decrease the likelihood of bugs_ (I wrote a bit about it here https://pscanf.com/s/351/, if you're interested), but of course only formal verification would ensure there aren't any.

> It is hard, today, to explain exactly what Lotus Notes was.

Whenever I try to explain what it does to a non-tech person, I'm met with confused looks that make me quickly give up and mumble something like "It's for techies and data nerds". I think to myself "they're not my target audience".

But I actually would like them to be, at some point. In the 90s "the generality and depth of its capabilities meant that it was also just plain hard to use", but now LLMs lower a lot the barrier to entry, so I think there can be a renaissance of such malleable¹ platforms.

Of course, the user still needs to "know what they need" and see software as something that can be configured and shaped to their needs which, with "digital literacy" decreasing, might be a bigger obstacle than I think.

¹ https://www.inkandswitch.com/malleable-software

Question regarding the pluggable js engine: I have an electron app where I'm currently using QuickJS to run LLM-generated code. Would edge.js be able (theoretically) to use electron's v8 to get a "sanboxed within electron" execution environment?

But joking aside, putting it like that I'd think I'd prefer the German/Codex way of doing things, yet I'm in camp Claude. But I've always worked better with teammates that balance my fastidiousness, so maybe that's my answer.

Oh I feel that. I sometimes ask ChatGPT for "a review, pull no punches" of something I'm writing, and my god, the answers really get on my nerves! (They do make some useful points sometimes, though.)

> On a final note: when building agentic applications, I used to prefer GPT (a year ago), but I can't stand it now. Robotic, mechanic, constantly mis-using tools. I reach for Sonnet/Opus if I want competence and adherence to prompt, coupled with an impeccable use of tools. I reach for Gemini (mostly flash models) if I want an acceptable experience at a fraction of the price and latency.

Yeah, this has been almost exactly my experience as well.