anyway, real feedback:

this is one of the longest existing unix service patterns ... what's this doing that the distro and usual suspects don't? (as a startup ISP in early 90's, we found what we needed already in BSD 3 distro)

to browse countless variants, swing by github, search terms like angel, daemon, dead man monitor, etc., although to be fair, most of those will long predate vibe coding, so this one may bring properties those don't

such a tool likely shouldn't be yet another separate SaaS, and the 'narrower' point is why: it should not feed that separate Saas (opposite of narrow), it should feed OTel, PagerDuty, whatever is your "broken things" sink, dash, and comms tree sender ... if you don't have that, do that first, then do this class of thing

* Results useful as they differ from pop theory, but mostly clustered enough you'd pick the language for other reasons.

https://www.noodlesoft.com/

It does lots more, but this bonus is great.

Whether selecting for ads or content, one had to figure different ways of getting after it, whitelisting was already off the table.

We'd found "structure" could work, but had to be fuzzy and adaptive as marketing shuffles pages around constantly, not just on the whole, but for A/B runs and personalized "experiences".

So that seems back to, hey, models could help.

(Though none of them, to date, attain as high success as latest hand coded "reader" tools, worth inspecting for how this could be generalized.)

Try to not get more upset when readers coach you than when the LLM rewrites you. Both happen, both are fine.

STRONG agree. Most devs, if they can see their costs, try. Even just putting it in a starship prompt goes a long way!

Observability supports effective, responsible, and accountable use, "cost controls" (as typically enterprise defined) supports bean counters. Ensuring Observability is recognized as a valid Control™ should be on every engineering executive's TODO list.

> the cache is the dominating cost reduction force

TY for the rich reply. As you're very aware of the cache issue, it might have been nice to have the word "cache" more prominent on your "Key Techniques" image (currently Lower Cost Models, Smart Routing, Spend Controls, Context Optimization; cache tuning buried at the end) or higher in the body sections.

It was mostly the image that prompted (ha) me to call out the marketing driven angle.

Sorry for the particular link, I'd grabbed whatever.

If one takes a minute, one can find contemporary newspaper articles. IIRC, many were about how people could suffocate.

https://books.google.com/books?id=Lm03AQAAMAAJ&pg=PA337&dq=t...

Contemporary women's health:

https://books.google.com/books?id=9-FXAAAAMAAJ&pg=PA71&dq=ho...

And contemporary survey of fears:

https://books.google.com/books?id=-RdAAAAAcAAJ&pg=PA107&dq=%...

Aaand, a 2023 takedown of claiming claims about historical tech hysteria:

https://www.wired.com/story/technology-predictions-history/

Both points suggest your subscription support was well chosen before.

https://illuminatingfacts.com/the-train-speed-panic-why-peop...

Reason that hysteria remains apropos now is that then, and now, we're figuring out how to deal with something beyond merely incremental change.

Irony is roads and sidewalks should have changed much more. We never did get around to good controls leading to zero deaths*, we decided a 100 people dead per day is a reasonable cost of convenience.

How much do our cyber traffic and cyber pedestrian controls need to change for everyman to get to drive AI? Car seats? Seatbelts? Air bags? Speed limiters? Pedestrian only living spaces? Driverless cars?

Many practical responses, likely a mix of things we haven't thought of yet, just as horses and horse drawn carriages didn't require most of them to coexist. Controls develop like scar tissue more naturally than they appear in advance.**

And if the better analogy for LLMs turns out to have been less like cars, more like flammable gas blimps, we'll figure that out too and tell cautionary stories for generations... but long before the stories are forgotten we'll have already come up with something more practical, faster, and – oh well – perhaps even deadlier per mile.

---

* Still working on https://en.wikipedia.org/wiki/Vision_Zero

** Not saying that's as it should be.

Anyway you'll likely get a more effective and up to date set of advice by skipping marketing pieces and pointing your favorite frontier model at /r/LocalLLaMA in deep research mode and asking it to synthesize the latest advice, or even try ideas out for itself and let you know what works for your own session history.

That said, this type of post needs to be read with product marketing context in mind.

> I think the most important aspect from it is the auto-routing

On the contrary, in white paper studies auto-routing is shown to destroy the single largest token cost they found, curiously not shown in their opening graphic. As they put it in their own words near the end:

"simple tuning of… caching settings… 50% reduction in … costs, with no observed quality degradation…"

They also aren't showing the harness called ‘pi’ which generally tops results (not only with ‘open’ models), and is consistent with Anthropic's recent "works better when we don't stuff 100k system prompts into context" about Opus 5. Recent models do better with less jank trying to prescribe behavior.

So then one wonders: why is this post written to say you should use a meta router but no mention that's offset by the cache busting, and show developers cost savings but no mention that Anthropic and OpenAI both have developer-facing utilization dashes already?

Perhaps it's so engineers can show enterprise procurement why they want to buy exactly what Databricks happens to have for sale.

Tell your PjM who should tell your PgM who should tell your PdM, all the PMs...

Maybe if "the business" sees it is true of LLMs, they might believe it's true of giving better context to engineers up front then giving them time to think and prototype (thinking tokens are an answer prototype).

I agree with one aspect of the narrative you're saying, his idea was held as being for «documents». To be sure, document was more abstract than document on a Mac.

Either way, I'm very sad it mostly turned into animated brochures instead.

But the reason styles were separate, the reason everyone talked about semantics, was always about the universal knowledge DAG and enabling machines to help us navigate and retrieve it. It's why so many of the early agents referenced navigation (Spyglass, Navigator) or the interconnectedness of things (Mosaic).

So, ok, big document “store”, but that's a lifeless noun. The information you are looking for, the dots connected for you surfacing what you didn't know, that's a living growing thing, an agentic retrieval savvy knowledge engine, distributed for curation and resilience.

The librarian was always to be a machine or many, on our behalf.

USER AGENT

not user, an agent on behalf of the user.

the web was intended to be usable by agents belonging to users. in our path of the multiverse, the main agent we think of happens to be live interactive browser. but that need not have been, nor will it be in the future, the dominant USER AGENT.

for a glimpse at one possible futureverse, check out what the home assistant community is up to, how they assemble then update the ambient information displays on their walls. their USER AGENTS are doing what most HN-style "hackers" dreamed of reading scifi as kids.

the alternative is all your in home information owned by corpos when the info should be from user agents not corpo feeds. if you want to vote this idea down, you might be a corpo. :-)

Comments URL: https://news.ycombinator.com/item?id=49202305

Points: 4

# Comments: 0

Not even brain teasers or IQ. Just: can applicant FizzBuzz? Verboten!

This is how, in a mass job application world, enterprises end up with >80% of coders, all making 6 figures, that can't FizzBuzz.

A hack to get the enterprise to assess is find the third party SWE test providers to big brand logos the board recognize, brands under the same risk governance / regulatory regimes, and ask "what do we know they don't?" First, the board know and respect those brands. Second, the question makes the board consider what they think about those in charge of the failing capabilities internally. This is something now both easy for the board to discuss, and obviously done by good references. Ok, let's try empirical evals.

Countless such broken record warranting beliefs clanging around 100k employee enterprises. McKinsey, BCG, PWC, KPMG -- all they need to do to justify 8 figure bill for the year is undo any one of them.