But here, again, we run on B200, the official weights with specific configurations that we have on vllm.

We will release our data soon. It's not just quantization, we are using NVFP4 from RadixLabs for GLM 5.3 but with our configuration and the OpenRouter harness for benchmark we are actually #4 in accuracy out of about 30+ providers. We run strictly on Blackwells (B200) on SGLang.

So its not just quantization, its the software, its version, its configuration, the exact hardware etc. They all matter.

I'm also the CEO of a new company - LunaRoute - so if you want private, fixed cost, we run our own server in the US type of DeepSeek 4.1 Flash check out https://www.lunaroute.com

If you want a trial, hit the contact us and write that you saw this post.

We also have GLM 5.3 (with vision!) and GLM 5.3 Flash - all included.

What happens if every turn in an agent loop goes through Anthropic’s Batch API instead of the normal synchronous endpoint?

The motivation was cost. Batch API is 50% off, which sounds very attractive for agent workloads: evals, background research agents, CI agents, unattended subagents, etc.

The result: it works, but it is awful for a single interactive agent.

In my runs, a one-entry batch usually took ~90–120 seconds to complete. That means a five-turn tool loop becomes a ten-minute interaction. Waiting two minutes for the model to decide it needs to run ls is not a good UX.

But that was also the point of the experiment. A single REPL turn is probably the wrong unit to batch.

The interesting version is fleet-level batching:

- many agents running in parallel - background subagents - CI/eval jobs - multiple harnesses sharing a local proxy - shared prompt prefixes that may benefit from caching

In that world, the batcher should probably sit below the harness as infrastructure. Existing tools keep using the normal API shape, while a proxy decides per request whether it should go sync or async based on latency tolerance.

One surprising observation: in my small, non-rigorous testing, Haiku batches often felt slower than Sonnet/Opus batches. I wouldn’t treat that as a benchmark, but it does suggest routers should measure this rather than assuming “cheap model = batch model.”

Repo is here:

https://github.com/erans/batching-harness

It is intentionally small: one Python file, a basic tool loop, local shell tool, stats panel, and minimal sandboxing.

The useful lesson for me was:

Batch API is terrible as an interaction pattern for one agent. It might be very useful as a hidden optimization layer for a fleet of agents.

Comments URL: https://news.ycombinator.com/item?id=47925427

Points: 3

# Comments: 0

Once the compromise point is preinstall, the usual "inspect after install" mindset breaks down. By then the payload has already had a chance to run.

That gets more interesting with agents / CI / ephemeral sandboxes, because short exposure windows are still enough when installs happen automatically and repeatedly.

Another thing I think is worth paying attention to: this payload did not just target secrets, it also targeted AI tooling config, and there is a real possibility that shell-profile tampering becomes a way to poison what the next coding assistant reads into context.

I work on AgentSH (https://www.agentsh.org), and we wrote up a longer take on that angle here:

https://www.canyonroad.ai/blog/the-install-was-the-attack/

You can have a single proxy and have multiple team members use the same one so that you can easily track sessions, token usage, spend etc.

- See Everything Your AI Does - get full logs (JSONL), summary of sessions including tokens used (input/output) as well as tools usage and success rates.

- Privacy & Compliance Built-In - redact or tokenize any sensitive information (regex based).

- Speaks OpenAI and Anthropic dialects so you can route (and translate) when needed between models and providers.

- High performance - passthrough is 0.1ms - 0.2ms latency. Logging and summary is off loaded to a secondary thread to allow maximum performance.

Feedback is always appreciated as well as stars on the repo :)

Comments URL: https://news.ycombinator.com/item?id=45663045

Points: 8

# Comments: 1

There are A LOT of businesses (even big ones managing money and what not) that rely on spreadsheets to do so much. Could this have been an app/service/SaaS/whatever ? probably.

What if these orgs can (mostly) internally solidify some of these processes? what if they don't need an insanely expensive salesforce implementor that can add "custom logic" ?

A lot of times companies will replace "complex software" with half complex process!

What if they don't need Salesforce at all because they need a reasonable simple CRM and don't want to (or shouldn't) pay $10k/seat/year ?

There are still going to be very differentiating apps and services here and there, but as time move on these "technological" advantages will erode and with AI they erode way faster.

Comments URL: https://news.ycombinator.com/item?id=44839717

Points: 7

# Comments: 3

Comments URL: https://news.ycombinator.com/item?id=12491179

Points: 3

# Comments: 2

The T series (and W series) are the work horses here with various screen sizes, disk options and RAM.

I personally use the X1 Carbon 3rd gen (X series) as it is comparable in specification to a Macbook Pro 13" (I have the one with the 16GB of RAM). It's a little costly but its worth while and will probably serve me for the next 2+ years.

Comments URL: https://news.ycombinator.com/item?id=7507027

Points: 1

# Comments: 0