This is my experience. Tests are perhaps the most challenging part of working with AI.

What’s especially awful is any refactor of existing shit code that does not have tests to begin with, and the feature is confusing or inappropriately and unknowingly used multiple places elsewhere.

AI will write test cases that the logic works at all (fine), but the behavior esp what’s covered in an integration test is just not covered at all.

I don’t have a great answer to this yet, especially because this has been most painful to me in a React app, where I don’t know testing best practices. But I’ve been eyeing up behavior driven development paired with spec driven development (AI) as a potential answer here.

Curious if anyone has an approach or framework for generating good tests

Anyway, as a result, I switched to Claude Code Max and I am equally as prolific and paying 1/10th the price. I get my cake and to eat it, too. *Note there’s a Cursor Ultra, which at quick glance seems akin to Claude Code Max. Notice that both are individual plans, I believe I’m correct you benefit from choosing those token-wise over a team or enterprise plan?

Anyway, you’re right Claude Code is less ergonomic; generally slower. I was losing my mind over Opus in Cursor spinning up subagents. I don’t notice that happen nearly as frequently in Claude Code itself. I think it has to do with my relatively basic configuration. CC keeps getting better the more context I feed it though, which is stuff like homegrown linters to enforce architecture.

All to say, Cursor’s pricing model is problematic and left a bad taste in my mouth. Claude Code seems to need a bunch of hand holding at first to be magical. Pick your poison

But I haven’t seen examples of products or software produced by these setups. Does something like Product Hunt exist for agentic coded things?

Worth noting here that I’m not a skeptic nor advocate. I work with coding agents daily and I see their utility. I also feel their pains. I’d love to lean in more but I don’t see tangible evidence beyond “more code”

Comments URL: https://news.ycombinator.com/item?id=47520857

Points: 2

# Comments: 0

I’ve been building out internal linters that enforce design patterns I want and raise common code smells (also note tools like eslint allow custom rules which are easy write with something like opus 4.6). The use case is a total refactor of react and fastapi apps. We are suffering from everything’s a snowflake syndrome and just want the same pattern employed across features.

This works pretty well when the linter has a companion agents.md file which explains the architecture and way about the world.

But to get the agent (Claude code opus 4.6 currently) to nail the directory structure and design primitives, and limit some doofus behavior, I still haven’t cracked how to make literally each line of code simple and sensible. And I haven’t figured out how to prevent agents from going out of bounds and doing weird things unless I catch it in review and add another rule.

This is a relatively new endeavor, but my gut is that it’s not much more time (linter rules and perhaps “evals” or a beefy agent review cycle) before I have bespoke linters in place that force what I want from our architecture.

Note that a huge bottleneck to all of this is that the codebase our current team inherited has no tests. It’s too easy to accidentally nuke a screen’s subtle details. It’s also really hard to write good tests without knowing what all of the functionality is. It feels like a blocker to a lot of large-swath agentic changes is a test strategy or solution first then a rigid push for rearchitecture or new design.

We tried to get as much mileage as we could out of Postgres before we switched to clickhouse and it was night and day.

But but but, in some scenarios it has been at the expense of my well being. It’s not good to take on more work and not let go of some of the things you’re currently doing. Moreover, finding “permission” from your boss to let those things go can be challenging.

I’ve found this works best when you and your boss agree on the problem you’re stepping into (not necessarily your solution). It may be that you need to stick your neck out and suffer for awhile for them to see your perspective.

When you’re on the same page about what you’re solving, a good manager will clear room for you.

P50 latency for a fastapi service’s endpoint is 30+ seconds. Your ingestion pipeline, which has a data ops person on your team waiting for it to complete, takes more than one business day to run.

Your program is obviously unacceptable. And, your problems are most likely completely unrelated to these heuristics. You either have an inefficient algorithm or more likely you are using the wrong tool (ex OLTP for OLAP) or the right tool the wrong way (bad relational modeling or an outdated LLM model).

If you are interested in shaving off milliseconds in this context then you are wasting your time on the wrong thing.

All that being said, I’m sure that there’s a very good reason to know this stuff in the context of some other domains, organizations, company size/moment. I suspect these metrics are irrelevant to disproportionately more people reading this.

At any rate, for those of us who like to learn, I still found this valuable but by no means common knowledge

You never knew. There are plenty of intelligent, well-intentioned software engineers that publish FOSS that is buggy and doesn’t meet some arbitrary quality standards.

Is this evidence that their hiring process is sound, and is it more a consequence of Linear being a rocketship? Perhaps if their retention number includes when a bad hire is let go, this is more believable that they are meeting their standards.

I’ve only worked at small startups, but usually “retention” means that no one has left for somewhere better.

It’s still unclear to me if python is too expressive for its own good, or if it’s so widely used, that it’s impossible to avoid nonsense

I can see both sides. We don’t have work planned more than a quarter out (a good thing IMO). Generalist SWEs make a good fit. But we think we need someone specialized in AI/ML. Unclear to me if that’s the case… and how to plan for it if we don’t want to explore concrete features we _might_ build.

I know this is a hot take, but so much headache saved down the road to “force” this stuff up front.