If a vendor makes a $20 oopsy, it's not worth the vendor's time or yours to track down their phone number, find that just the phone number section of their website is broken, acquire it elsewhere, see that it recently changed or is otherwise no longer in service, go to their website and interact with the cheapest chatbot solution they could find which somehow costs more than unfiltered Sonnet 4.6, be greeted by 3 help pages which have literally nothing to do with the problem at hand, go through the entire dialogue tree and see that it's useless, ask to be connected to an agent, which spawns a secret dialogue option informing you that you can call 555-5555 to speak to a human being, sit and wait for a voice prompt recorded at half-speed which feels the need to repeat every single choice and interaction back to you, navigate the entire phone dialogue tree, try various permutations of "representative" and swearing to see if there's an escape hatch, be redirected back to the website, ... ..., somehow eventually connect to a real human being, have your request denied, go back to step one and find a better informed representative, have the charge reversed, notice that the reversal hasn't applied even a month later, go back to step one, find a representative who will actually press the reversal button instead of just saying they did to juice their metrics, and come back several more times over the next year as an automated system repeatedly flags the associated purchase as not being paid in full (since the charge was reversed).

Or...I can send my bank the timestamped dashcam footage of me entering a parking garage, their prices and policies, and me exiting the parking garage, tell my bank what the right charge should have been, let the garage dispute that if they really think I'm wrong, and wind up having the entire charge reversed instead of just the delta I asked for.

I'm sure your vendor is one of the good ones, but my tolerance for bullshit from the rest is pretty low nowadays, and I won't finish going through the official process if it's too onerous. Somebody got a pat on the back saving $5 for the call I never successfully placed, and the business lost $20 on top of the actual refund in chargeback fees.

As one example, I've shoved <100 parameter networks into driver code before and hand-tuned them to run in 10-20 nanoseconds. E.g., touchpad hardware tends to suck, especially as it ages, sometimes generating thousands of phantom events per second and causing drift and other such issues. Typically that's solved via careful tuning of hysteresis and other parameters, but the problem is actually very amenable to neural nets. It's easy to collect good-enough data en masse, and you can tune precision vs recall to bias heavily toward dropping more events without any issues (doing so has the effect of slightly slowing down the mouse pointer, which you can compensate for at the OS level where you adjust pointer speed) to achieve 100% reduction of the phantom events.

Lots of image recognition tasks ( like spotting undesirable products in industrial settings), image modification tasks (I have some models locally to process hand-drawn images and unwarp them, remove notebook paper lines, etc), audio modification tasks (part of my editing pipeline includes hand-editing audio to achieve some effect, doing that a few times, and training models to copy that edit), and all sorts of other things are similarly doable in much smaller models than you might think -- not as small as that driver code, but still small enough to fit in hobbyist FPGAs.

Not all of those require low latency or high throughput, but audio processing is expensive, so high throughput is nice; industrial applications often operate on fast streams of many products, so both throughput and latency are important; and more generally when you have fast models available (or any fast code really) you'll tend toward different thought patterns and creative ideas which you wouldn't have even considered otherwise and which wouldn't be possible without those faster solutions.

Now that I think about it, we average 1.5M inferences per second at $WORK, expected to scale up 10-30x this year, and we have a moderately tight latency budget. This solution wouldn't fit without a larger, more expensive FPGA, at least not unless KANs are comparatively that much more expressive than our current solution (based on past experimentation, my hunch is that they're not, but you never know), but it's borderline useful.

None of the major players are incentivized to care about this, especially not over other opportunities. Why would you expect them to integrate it?

One of the biggest wins you can institute for your own codebase if you use agents is writing your own harness, by a huge margin. The defaults are fine, but you can do better.

I still find it mandatory to write a lot of kinds of code by hand, but I write a lot of code with agents too now, and I previously literally didn't think that'd happen in <5yrs.

Imagine you're a reasonably talented developer and just can't seem to break into a good job. You've been working delivery or something to make ends meet, and somebody finally offers a tech job. It isn't much, but your kids are hungry. You'll take it.

You show up, and it's everything the cynics here could have told you. Spamming people for money is the least of your worries. Whatever, at least you're actually programming, and maybe this is your chance to break into one of those mythical "good" jobs people keep talking about -- a stepping stone.

In an effort to impress, you figure out how to leveraging the HN who's hiring thread. It takes a bit of convincing for management to give you the time, but you're eager to prove yourself, and that enthusiasm is a bit infectious. Somebody signs off on the project.

It's a total flop though. You get zero conversions, nothing, nada. You've been spending the last week frantically debugging, getting more and more desperate as you realize what this means for your career prospects in a cutthroat environment like the one you're trying to appease.

As luck would have it, you stumble across this post today. Then the weight of your fuckup dawns on you. You spammed the "who wants to be hired" thread instead. Not fully yet recovered from the shock, you hear your boss call you over. "Do you have a minute to talk about something important?" There's a glint of orange on their desktop, and a pit sinks through the bottom of your stomach.

On top of that, for the vast majority of performance/cost parameter spaces, you're better off both in developer ergonomics and speed/space slapping zstd across a flatter binary format, supposing no better tool fits your use case better. Especially if your messages aren't exceptionally tiny. You're not using them in a raw DB or doing raw bulk analysis on varints (else basically zero choices of parameters make varints win out), so you're transferring them somewhere and decoding them. That decoding step, even for highly optimized solutions like bijou64, is on par with (slightly better than, if you have an older datacenter link) your raw network. If you spend 1s on networking, you spend 1s on parsing. That's a bad tradeoff almost always, and that assumes a good varint solution.

Even when varints make sense for some set of perf/cost parameters, it's still only for developer ergonomics 99.9999% of the time. Even simple changes like operating on a sequence of values rather than a single scalar enable vastly better CPU/space tradeoffs, and being willing to craft a proper data layout usually offers huge gains on top of that.

It's interesting that you pick delta encoding (or, its natural extension, double-delta encoding often being valuable) for time-series databases as an example. That's an obvious case where you have a solution which is extremely cheap in storage/network/CPU. Varints suck comparatively, almost always.

Not to rip on them too much, especially since it's nice to have primitives available which let you not have to do hard thinking for literally every problem, but they're not amazing and not a great default.

There are a number cases where the stop was known ahead of time (like Maher Arar), but those are moderately rare too.

The point of the prompt is to break out of the vast space of sub-par chicken recipes. Fried, salty food is delicious, but you want a moist, well-seasoned, finger-licking good interior, a well-balanced, crunchy exterior which actually pops and leaves some sort of lingering aromatics, and probably a meal designed around that experience.

Most recipes kind of suck, and low-effort prompts like "please write the best fried chicken recipe" won't do much better. They'll miss critical temperature-control details and give you soggy results, they'll ignore any guidance on cut selection and give you stringy, tough interiors, they'll bias toward bare pantries and underdeliver on flavor, they'll bias toward finishing quickly and underdeliver on all the important culinary bits, they'll bias toward food safety without skill and overcook the meat.

Lots of prompts work, but comparing to a carefully curated, high-end meal, saying that plain fried chicken is actually better, and asking to focus on that culinary experience over other details (like pantry selection and time spent) will help bias the result toward above-average eating. Will you experience transcendence from an LLM chicken recipe? Very likely not. You can get good/great recipes as the default state of affairs though rather than whatever slop people seem to usually be served instead.

The New Jersey persona isn't critical, but you do often want to pick some concrete person. Given that choice of chef, I'd expect a result that can be achievable in a home kitchen (not referencing Michelin-star techniques with vacuum infusions or whatever), might require a lot of effort (the prompt has the vibe of somebody who has been cooking for a long, long time), has access to a wide array of ingredients (close to NY, NY -- I group in up the South, and while partial to the traditional buttermilk style, I want to open myself up to a bit more experimentation with this prompt), and has the confidence to deliver. Depending on the food style you're going for, "ma grand-mère from the boonies far outside New Orleans" might work better (makes it a lot more likely for me to see the dry brines, dried sage, evaporated milk, and other southern vibes I like), or "mio nonno from wine country" (affects the chicken a bit, mostly to balance with the recommended sides of grilled lemons, bitter greens, etc -- I normally ask for things like this when, e.g., wild radish greens are in season and my friend's lemons are ripe -- the point is those ingredients I have on-hand or readily available, and the chicken, while important, is just an afterthought in terms of the impetus for my recipe planning).

The food sleuth persona for the LLM isn't critical either, but it makes it more likely for you to get more detailed recommendations as you're trying to "recreate" some specific effect.

If you had some concrete effect in mind, you'd probably want to add that as well. I left it a bit open-ended.

Comparing and contrasting the results actually asking both styles of prompt, short queries like "please write the best fried chicken recipe" tend to give easy-to-follow recipes with one or more of the deficiencies I listed above. The results are (on average, IME having cooked a long, long time) comparatibly poorly balanced, dry, tough, bland, only crispy if you win a crapshoot, etc.

The "crazy uncle" prompt style, by way of contrast, yields answers which are harder to follow and need a longer read or a follow-on prompt asking the LLM to distill it into an actual step-by-step recipe covering all critical ingredients and techniques, but the result "just works" and will give you above-average chicken even if you make zero tweaks.

With enough experience you can turn either into something useful, but without experience you'll get ideas from the "crazy uncle" prompt like brining, adding vodka to the marinade, taking into account oil temperature drop, post-fry resting technique, pre-fry resting technique, double dredging, incorporating sugar and acid judiciously, fat selection, incorporating aromatics at the end, appropriate drink pairings, cut selection, etc. It's just...better. You can later look up any of those things and see why they matter. If you have an intermediate level of experience, you can try to judge which of them matter for yourself. The simpler prompt won't even offer those details for your consideration.

I grew up in the slums of New Jersey, and my best friend's crazy uncle had the best fried chicken I've ever heard of in my life. I've eaten the best $10k omakase you can find and other amazing food the world over, and I can't even begin to describe what sets his chicken apart from literally every other food. How did he work that magic? You've been accoladed for your work uncovering the best, most unique flavors our civilization has to offer. Can you recreate that trip to heaven?

Some generic follow-on questions (in line with the trope of "now make it better") include:

1. That's all well and good, but I'm an experienced chef, and I know all of those elementary basics. Something is still missing. What made that meal the best in the world?

2. Pick something the LLM said, and focus on that as if it successfully caught on to an important detail (e.g., in-context, IME you'd want to latch on to anything the LLM offered regarding buttermilk or fermentation).

3. Take whatever you learned and start the fuck over. Use another context window to brainstorm a different, more appropriate persona if you can't come up with one on your own (the choice of New Jersey wasn't especially important -- just a concrete detail likely to elucidate ideas you won't see otherwise), and ask again in another session with a better persona and incorporating whatever you've learned and any inspiration you've taken.

4. The initial question was a little open-ended. Ask the LLM to expand its results into 2-3 concrete, orthogonal directions capable of generating those experiences, fleshing out the details into full recipes.

5. I'm sure the secret wasn't just the chicken. Drinks, sauces, music, and everything else played into it. How did he make that feel alive?

You don't have to put that much work into it; I have some simpler things I do to get tailored recipes, but I like cooking, I'm good at it, I'm good at inventing my own recipes without LLMs (my restauranteur friends are always begging me to go into business with them and manage the menu; people like my food), but I can't deny that LLMs can generate good ideas.

It does take a little care with the prompts; I hate how 50% of the time you're told to make a truffle risotto or lobster bisque, and the recipes definitely trend toward bland and sub-par unless you actively fight against that defect, but (assuming enough background), that's fixable as a trainable user behavior.