This is of course anecdata. I know plenty of outliers, too. I know a principal engineer who uses many multiples of the number I quoted above. I am sure we also know many people making do with much much smaller budgets as well, via all kinds of well-discussed methods.

But, "$500-$1500 per month per full-time developer" is just kind of the personal mental baseline I use when making my decisions with regards to thinking about whether any of this makes any economic sense.

The Sparks admittedly are kind of anemic: 273GB/sec is the same bandwidth as a midrange 4060, although (depending on how you configure things) you can effectively have much greater bandwidth by connecting them.

Compared to 1-2 years worth of LLM tokens for a full-time software engineer making $100K+/year, a one-time spend of $12K for 4 Sparks for on-prem private LLM inference starts looking reasonable, particularly if privacy is an important consideration. It starts looking even more reasonable if running something like a private cloud to service multiple developers because then you likely need less hardware per developer.

(Also, it is going to be a long time until RAM+GPU prices return to what we used to call "normal." If ever. I am not endorsing the current state of affairs and I am not saying you wrong to find it insane, but it is definitely the new reality)

I'm usually running about ~4 instances of it locally in separate Git worktrees, which (in our setup) means I'm also running 4 copies of the whole stack: web, frontend, Redis, Postgres, etc. These are rebuilt and torn down multiple times per day. Lot of frontend assets to compile.

Additionally, like many people in corporate environments, we do of course also pay a performance tax thanks to the security software we need to run.

So perf does matter for us.

We could, of course, alternatively... just run cheap shitboxes locally and have all our dev environments in the cloud. Certainly appealing in many ways from a cost perspective and you don't even need to give up DX really.

However, in the next ~5 years, as "actually useful" local LLM inference starts to become genuinely useful (for software engineering, this seems to roughly mean "the ability to run 28B+ models at usable speeds") then having local compute horsepower might start to look more attractive again. Even if it's not an absolute winner on cost/utility alone, the data privacy and cost certainty have value to many.

In general, the reason why modern CPUs are so complex is because the gap in performance between CPU and memory has grown massively over time.

In the old days, something like a 6502 was running nearly synchronously with RAM.

That gap has grown massively over time; a modern CPU is orders of magnitude faster than RAM. So they have to jump through a lot of loops to avoid simply idling 99.99% of the time while they wait for some new data or instructions from RAM. On-CPU cache memory is one answer. Branch prediction and speculative execution are others. As you may imagine, speculatively executing code based on branch prediction is very complex because you must roll back any side effects from that execution if your prediction turns out wrong.

Example:

   # assume `i` is a value stored in main memory
   if i == 42
      j += 1
      k -= 1 
      l = 666
   else
      z = 123
      q = 5879873

That's why we can't just take an R2000 and scale it up to 3ghz. I mean, we could, but it wouldn't work very well unless we also had low-latency 3ghz main memory to pair with it.

But when humanoid enemies behave in plainly stupid ways it's a real immersion-breaker for me. I've been gaming a looooong time, so I'm quite adept at the necessary mental gymnastics to enjoy stuff anyway... but... still... games could be better here. (And by "better" I mean "more fun")

    I tried the qwen3.6-27b Q6_k GUFF in llama.cpp 
    and LM Studio on my M2 MacBook Pro 32GB machine 
    last week, and I barely get a token a second with either.

(not sure of exact diagnosis/fix, but definitely look in that direction if you're still having this issue when you give it another shot)

Also, there are two stages - prompt processing, and token generation. Prompt processing is notoriously slow on Apple Silicon unfortunately. If you have large context (which includes system prompts, lots of tools loaded by a harness like Claude Code, OpenCode, etc) it can take minutes for prompt processing before you see the first output token. On the bright side, the tokens are cached between turns, so subsequent turns won't be so bad.

    It's highly improbable that the US government has a secret team inside Anthropic and OpenAI manipulating their training regimen.

One: it would be relatively technically trivial for $GOVERNMENT_AGENCY to just monitor all the prompts + context we send over the wire to OpenAI/Anthropic/etc. That's a goldmine of sensitive personal and corporate data, no secret team needed (although, the LLM providers obviously would need to cooperate)

Two: Rather than secret infiltration teams influencing model training I think what's more likely on the training side of things is simply self-censoring by the LLM providers, so that they don't risk angering the government.

I highly doubt that China has government interlopers, secret or otherwise, inside Qwen's training team. Nonetheless, "sensitive" issues like Tiananmen Square are censored. I would imagine that much/most such censorship in China is self-censorship that doesn't leave a legal/paper trail. That's what we're in danger of seeing (more of) in America IMO.

Open-source model inference providers (who do not have to bear the cost of training) seem able to do it at much lower prices.

https://www.together.ai/pricing

https://fireworks.ai/pricing#serverless-pricing (scroll down to headline models)

Of course, it's possible that they are burning through investor cash as well, and apples-to-apples comparisons are not possible because AFAIK Google does not mention the size/paramcount for 3.5 Flash.

But if the prevailing wisdom is true, I think it's actually encouraging. It suggests that OpenAI and Anthropic could perhaps, if they need to, achieve profitability if they slow down model development and focus on tooling etc. instead. If true that's probably good news for everybody w.r.t. preventing a bursting of this economic bubble.

...my opinions here are of course, conjecture built on top of conjecture....

You can get part of the way there by Sublime Text to run queries and browse results. In Sublime, it's easy - you can create a custom "build tool" with (essentially) just one line of config... you're basically just piping the current file's contents to command-line PSQL: https://www.youtube.com/watch?v=tPd4m3PLVqU

Then I use the SublimeAllAutocomplete package. It works just like "regular" Sublime autocomplete except it gives you autocomplete suggestions from all open files, not just the current one - so if you have your DB schema dump in another window it will use that: https://github.com/alienhard/SublimeAllAutocomplete

Obviously, that doesn't really give you the smartest autocomplete ever but it's pretty productive for me. Tons of room for improvement though.

I actually enjoy early-morning coding (before the rest of the world wakes up) more than late-night coding now!

What I'm saying, though, is that the people around them at the bar are far more likely to be able to assist than a conference organizer who's not physically there.

"The point is that conferences should have a policy for how to deal with these kinds of situations. Those policies are probably going to depend on a certain amount of judgement from the conference organizer."

I totally agree that there need to be policies. But policies are just words, and they won't get the job done alone.

What will get the job done is looking out for each other. In social situations, particularly bars, we should all be looking out for each other, especially women, since they're disproportionally the ones on the receiving end of predatory/harassing behavior.

If she went to the bar with a group, every single one of them ought to have been looking out for each other. And yes, I've been in scores of similar situations where I and others have looked out for the welfare of others. You don't need to be an obnoxious "white knight" about it; you don't even need to be overt. Creeps tend to look for girls whose friends aren't paying attention.

Example: Your female friend is caught in a conversation with a potential El Creepo. Play dumb and introduce yourself to the guy in a purely friendly way. Heck, maybe buy a round of drinks. Just knowing that somebody noticed him will often nip things in the bud.

On one hand, running a conference/convention is difficult enough without being asked to referee personal conflicts. It almost inevitably devolves into a he-said/she-said kind of argument.

But on the other hand, if we take that kind of an attitude, that's essentially a signal for predatory men to go ahead and harass women (or worse). That's pretty much what predatory men have been doing since the beginning of history - acting with impunity since claims of rape or harassment are almost impossible to prove if there are no additional witnesses or physical evidence.

What we've done is to stress proper conduct before the event, and if there are repeat complaints about somebody they're removed from the community permanently.

It's not a perfect process and we've given some "second chances" to people that we've later regretted.