For example, most vision models don't need 256K context length for modes like Qwen3.8-27B when all you care about is single-image captioning, so you can technically save on KV cache. Video reasoning does require that context length, so it's a different set of deployment parameters that need to be enabled.

All of this to say that the providers that offer these models, are simply using vLLM / SGLang, and mostly cater to the text inference use-case (coding, etc). Vision always seems to be a bit of an afterthought.

If you’re building vision-native apps, there are so many footguns in vLLM/SGLang serving configurations, let alone the routing/orchestration in providers like OR, that leave the user more confused about the model’s capabilities.

[1] https://www.vlm.run/gateway

We’re building the inference platform for visual intelligence. We’re a deeply technical team of veteran AI / computer-vision engineers (20+ years combined, MIT/CMU PhDs) who’ve shipped production ML infrastructure across autonomous driving and LLMs.

We just launched the VLM Run Gateway (https://vlm.run/gateway), a unified API that serves open-weight VLMs, embodied VLAs, ViTs, served across a fleet of GPUs and clouds. That's exactly the infrastructure problem this role will own.

If you’re interested, email us at hiring@vlm.run with your GitHub profile and link to projects you’ve recently built - especially with docker, k8s, GPUs.

Comments URL: https://news.ycombinator.com/item?id=49365625

Points: 6

# Comments: 1

I get that in principle that there's no retention, but these are powerful models that can comprehend, paraphrase and summarize your logs for the sake of "product" improvement. Who knows what's collected here.

Posts like this just reminds me of the end to end demos AV companies built in the early days using a single camera - only to realize that it’s harder than it looks years later into development.

SOTA 80% means a practically useless robot. What are they really imagining their ICP to be here?

In either case, here you go, it's public now: https://github.com/vlm-run/mm.

Use Gemini or some local VLM to do this way more efficiently. We spent quite a bit of time on video understanding, and Claude will just burn tokens.

Check out this library: https://vlm-run.github.io/mm/

You can swap models and try out different encoding methods for videos (https://vlm-run.github.io/mm/encoders/#video)

Comments URL: https://news.ycombinator.com/item?id=48575159

Points: 2

# Comments: 0

In all honesty, opencv has stood the test of time and I’m certain newer LLMs will likely not attempt to rewrite it from scratch.

P.S. I’ve been a user since the IplImage days, circa 2007, and I’d still consider using it over most CV libraries today.

Seems like a concerted and distributed effort from the entire Anthropic team every time to get this on top of HN.

I’d love to see if we can get GPU access within these runtimes, that’d be awesome.