What signals are you using for this assessment? Are they indicating embarassment? Do you honestly see their customers being concerned over this?

Like lion tamers in a circus, Anthropic and OpenAI thrive on the theatricality of how scary their pets appear and so they play it up by prodding them to growl and snap at chairs and then mug for the audience every time it happens. And to their delight as performers, the audience gasps and cheers each time.

They want to make their pet seem the most powerful and unpredictable and they want their audience to believe that they're holding it back from catastrophe but only barely and only because of what unique talent they have.

This is not embarassment.

Later, when you want to try the return, a black box algorithm asseses your transactional value to Amazon and decides whether your concerns are worth attending and to what degree.

Maybe that really is better than whatever you were used to in your own market, but it's a profound regression on the traditional retail experience for most of us here.

You need to let go of the idea that this is something LLM's can do. They can't. At best, they can bias towards rules conformance with more or less likelihood, but coverage and conformance both go down super-linearly as you accumulate more rules, more context, and more output in a session. That's simply the nature of how these tools work and you need to engineer your workflows around it if you want to use them.

If you absolutely need some rules enforced, you need to adopt some framework for validating those rules that then rejects, reprocesses, or repeats any session that fails to satisfy them. In the best case scenario, this is some traditional deterministic validator (like a linter, compiler, analyzer, exhaustive test suite, etc in coding) but if you need to process in stochastic space because its something rich and ambiguous like natural language itself, then you want to dispatch a swarm very narrow, task-focused subagents that each validate against a very constrained subset. (And prepare yourself to have those to fail sometimes too. LLM's are noisy and cannot deliver strict rule enforcement on their own.)

When a human does it, it's identifying. Like the timbre and dynamics of their spoken voice itself, It distinguishes them from the dozen other people you're working with on the project and the thousands of people you encounter through your days. It's signal

But when we have a handful of popular models, and they answer every question everybody has, and get quoted and forwarded everywhere, and are used to reformat and rephrase personal communication... that signal becomes noise.

Rather than voices disinguishing sources in the cacophony of our lives, everything and everyone starts to sound the same, and we lose key information that we're biologically and culturally accustomed to relying on.

Some people are likely unbothered by this in the way that some people are face blind or colorblind, and so don't see the problem. But as we see in discussions like this, many many people do get bothered by it, even if they don't yet have the insight as to put their finger on why.

You can't trust the agent, let alone its harness, to oberve any particular directive you give it, so "md files" provide no meaningful protection for anything important.

But users are broadly reckless and naive and commercial vendors are exploitative and irresponsonsible, so the vendors take advantage of what they can get away with for as long as they can get away with it.

Use a tight sandbox, and join the chorus loudly when others press on vendors to be make user safety an earnest and hard-to-abandon priority.

Among the most critical realities to consider exist outside the product definition entirely, and have to do with the environment in which it will exist during and after development. These are things like plausible scaling curves and limits, code lifetime, team size, deployment options and preferences, etc. Others are things like available runway, team fluency and talent, available tools and licenses, etc.

One of the big eyeroll moments of the last 15 years or so was when folks started blindly cargo culting global-scale 10,000-engineer enterprise practices on projects that would obviously only ever be touched by 1-2 technicians and deployed on a far far far more modest scale. Instead of actively considering any of those environmental requirements above, "engineers" would try to treat every project as though it would someday service 10,000,000 DAU under the hand of dozens or hundreds of churning technicians.

Akin to the joke about some Americans who see themselves as "temporarily embarassed millionares", making foolish choices at their own obvious expense, many engineers during this era would seem to see themselves as "temporarily embarassed Facebooks".

In countless cases, this was simply very dumb and very wasteful and at best amounted to inadvertent (or perhaps intentional) resume-padding for the people involved, while their projects bloated and sagged and lurched under unwarranted complexity.

Sometimes (in fact: often), just doing the thing well with simple, clear, stable tools -- and nothing more -- is the right choice.

If you're choosing to rely on a strict, exhaustive validation process to approve your work product without concern for how it's produced, you shifting focus to several critical cost centers:

1. Exhaustive validation is not free and its cost grow super-linearly as your work product gains complexity -- both on its tested surfaces and in what inefficiencies it develops internally. As you invite more frequent commits of more uncoordinated code, you're running your QA process ever more often on ever more heavy code, covering ever more work surface area. This is non-trivial cost and potentially catastrophic risk.

2. AI costs scale with token requirements and project complexity. As you allow uncoordinated code to accumulate within your product, you super-linearly increase the token consumption required by AI agents attempting future commits and analyses as they digest the increasingly chaotic/uncoordinated code base, both in raw input tokens and the reasoning tokens used to strategize plans. Even if you believe that your AI agents will be able to keep up with this expanding burden indefinitely (a questionable assumption!), you're incurring another growing, super-linear cost.

3. If you're relying on frontier models to make this workflow feasible at all, you're at risk of uncontrolled cost ratcheting if and when your code grows so speghettified that only those models can maintain it. You may have economics that work today, but the model providers you use are positioned to squeeze your margin aggressively once you trap yourself within their ecosystem.

Now, it's possible that an adept team can figure out how remediate some of these concerns by appropriately tasking agents to reduce project complexity and efficiency independent of new feature development. But the overall workflow here remains uncharted territory, where few people are going to be getting it right, and many will find themselves being painted into a very expensive dead end.

TLDR; do what you want, but all the traditional concerns of technical debt management and vendor exploitation risk are at play here. They've just shifted around in a way that makes it easy for inexperienced or myopic people to think they've somehow escaped them because the underlying technology itself is so novel and unfamiliar.

In general, what you're talking about here is something that businesses have been able to do for decades by delegating their tasks to outsourced suppliers. The lesson, then and now, is that it works until it doesn't and that once it stops working, you're screwed.

So be careful -- the revolution you've promised is not what you imagine it to be!

What a generous euphemism. Depending on the topic, it's not unusual to see Google AI search referencing links that indicate the exact opposite of what it presents in its prose.

So many of these tools are presented as if they're doing something you'd intuitively expect of a human operator, like collating search results and then summarizing them in this case, but the actual operation of them is so alien to us that our intuitions don't apply and these presentations are all but fraudulent.

Because it's presentation cites references, and because those references sometimes reinforce the expression of the AI prose, we grant it trust, but it doesn't use those references in the way a human would and so the relevance of them being cited is radically different (and weaker) than we're accustomed to.

The closest example in human behavior might be the rushed, naive student who just pastes citations into an already written paper at the last minute, but even that's an anthropomorphism that misrepresents the alien ways in which the tool works.

These data centers are specifically being scouted for communities whose governance is too weak to negotiate for some "sizeable share of their profits" and too ill-prepared to have suitable environmental regulations on the books already. The Ivy League sharks planning these buildout initiatives are sharp people who are looking out for the interests of their employers and know how to pick locations where they have the best opportunity to exploit locals unprepared for their kind of esoteric deal-making, political lobbying, and lawfare. They'd be failing at their job if they did what you're suggesting, and that's why we don't really see that happen.

For any benefit to national or global society AI data centers might provide to someone, the buildout looks a lot like the dirty and exploitative stories of rail expansion in the 19th century. That rail infrastructure proved a good thing for the US, but that doesn't mean the process of making it happen was honest or good for the people immediately affected.

Meanwhile, golf courses are a traditional green space where people in a community gather for both work and leisure. They're not ideal themselves, but they at least provide some benefit against which their negatives can be weighed.

If all you hear from critics of data center building is water use complaints, that's strictly because you've chosen not to listen to people.

Gradually over its recent booming years, software work went from one of several practical engineering refuges for curious tinkers and puzzle-addicts to a career path for financially ambitious bright people akin to finance, law, or medicine.

Many people carrying a "software engineer" title now never really enjoyed that part of the work at all but were suitably clever and responsible to accomplish what modest ends they were tasked to by their very generous employers. Mostly (but not entirely), those people are the ones most eager to have AI agents shield them from rigorous design and puzzle work and enable them to leverage their innate cleverness more lazily. They never really internalized the coding and engineering principles of the industry and so can't foresee what might be down the road for them with this technique, especially when they're surrounded by peers with the same mindset.

> AI isn’t always faster.

It is when coding was an extremely frustrating and high friction experience for you in the first place, as is the case for many who work among us now.

I understand you're excited about the tool, but for the sake of earnest discussion here, maybe commenters like yourself can tone the hype down to plausibility?

Claims like this are just nonsense. It's not how product development works.

How do you even have so many bugs left to fix if the tool is so fast and productive? Surely, you didn't have a backlog of tens of thousands of bugs that you're still chewing through? And of course, the volume of new bugs much be minimal since the AI-composed additions introduce "no problems so far". If it works like you say, which we'll accept in good faith per HN guidelines, you must have exhausted your backlog long ago.

And if you've indeed exhausted your bug backlog long ago (incredible!), you're left to talking about shipping "10x as many features". Yet no product has a limitless capacity for features. Nobody would want to use software so bloated and churning that was gaining features at such a pace. And who is designing and specifying them so quickly anyway? If it works like you say, which we again accept in good faith, you must have stalled out on your feature list long ago.

If the AI indeed allows you to "[ship] 10x as many features and bugfixes", and we take what you say in good faith, then one of the following seems to be implied:

* you've fixed all your bugs and blew through your mature feature designs already, leaving your AI agents sitting idle for all but a few hours a week, while you're bottlenecked on feature design and your software product is bloated beyond imagination

* your coding productivity before AI was absolutely glacial by industry standards such that "10x" productivity for you is actually much closer to "0.5-2x" for others

Any insight into which of those it might be?

Scrubbing that away from writing with AI has the same effect as would running everyone's speech through an anonymizer.

Some people may be blind to that sort of thing entirely, much like some people can't recognize faces, but for most people it just creates a sense of noise and discomfort when everything everywhere is written in the same style. And that impression is only made worse when that increasingly uniquitous anonymized style is so painfully average and repetitive.

If there was a tool that let you edit select characteristics of your writing while preserving your voice, the way a professional editor might help a professional writer, that might be amazing for helping people communicate better and more easily. But that's not this. This is loud, articulate noise.

But history (and world-awareness) shows that those periods don't last forever, so having mature decentralized technology ready and warm for periods of crisis or devolotion is hugely valuable in the long term. It can be hard to maintain commitment to maturing and seeding that kind of technology when there's not yet a pressing need, exactly because it's hard to gain enough traction to overcome the relative inconveniences. It's admirable and important work regardless.

This is a trap.

It's the optimistic fallacy that poisons all "consumer scale" machine learning products and what's going to effectively ruin these models as they keep chasing it in the same way that web queries were ruined, social media feeds were ruined, and media recommenders were ruined.

For the vendor, optimizing metrics across their whole user base, they always see positive technological progress as their system gets better at making assumptions and accumulating user engagement scores in aggregate. But for the individual user, most of which has some weird tail intent/interest and some of whom have many weird tail intent/interests, the experience quietly but catastrophically degrades. Output/results become more generic, more divergent with the underspecified "weird tail" intent, and more stubbornly hard to ever wrangle towards that "weird tail" altogether.

We've been watching this cycle happen for 20 years now and it's proving hard for anybody to escape because it works so well for the trillion dollar company driving it forward. But while each step might feel ergonomic and welcome to individual users, there's a frog boiling enshitification at play.

In pursuit of output quality and capability (rather than simply the vendor's user count), what we need rather than "makes better guesses" is "presses for more clarity", even where it feels kind of annoying.

Even among human professionals, one of the first hurdles of breaking out of junior tier work is gaining the confidence to press your colleagues and clients to be more specific in their thoughts and expressions despite their desire to have you do it all for them. But they're often coming to you with incomplete, muddy, and conflicting ideas for which there is no safe and correct assumption that you might just run with, and it's your expertise (i.e. relevant "intelligence") that's critical to bringing attention to that. To achieve professional progression, you need to learn to do that and to not just optimize appeasing the ambiguous client/colleague today in exchange for mutual expense tomorrow. To avoid enshitification, which is probably not possible, we need these models to be learning that too.

* some sincere people with very extreme takes,

* some trolls that masquerade as the above, to bully others over their credulousness and lack of guile, which is distinct from sarcasm,

* some trolls that insincerely speak anything that earns engagement,

* and more and more bots that mimic the above

So sadly, the answer to your question is generally yes.

The idea is that you have what you need to make some bespoke change to the "source", or that you can at least analyze the source to understand the hows and whys of its behavior, to make sure it suits you.

Do weights provide either of those qualities?

Maybe reality isn't represented by the git logs, but according to them, you appear to have composed, reviewed and merged 131 commits affecting 58 files and 6000 lines -- a task that you say amounts to "replacing the engine while driving" -- in the span of one week.

If that's accurate, of course your reliance on AI is at fault here. It invites you to mistake velocity for quality and conflate test satisfaction for completion.

Without AI, this effort would likely have taken much longer and an experienced team would have approached it with due meticulousness, being especially stringent in review. Absorbed in it, the work would have weighed on their unconscious while they slept, with "Ah shoot, did I consider XYZ?" moments striking them in the shower. Ideally, familiar with the specific contributors (i.e. you) and their style of work, the reviewer who later consider merging the work would have ideas what to watch out for and would spend extra time looking at the details you might not have fully considered. It's a whole decades-matured craft practice that can do a pretty good job of making sure that a refactor of this scale doesn't land in main too broken.

And while that version still probably wouldn't have been entirely flawless either, or might never have even been attempted because of its greater calendar and attention burden, that doesn't mean that using AI wasn't responsible for why and how this version is broken.

This is exactly how AI makes things worse and why many people are wary of relying on third-party projects that embrace it too blithely. If you're going to use it, that's great, and maybe it will help you keep landing big features more quickly than otherwise -- but at least accept responsibility for the tradeoffs that you're inviting when you do so.