Hacker News: sethkim

A classifier is not a classifier is not a classifier

sethkim — Sun, 27 Sep 2026 18:04:15 +0000

Article URL: https://sutro.sh/blog/a-classifier-is-not-a-classifier-is-not-a-classifier

Comments URL: https://news.ycombinator.com/item?id=49869159

Points: 3

# Comments: 0

New comment by sethkim in "LLM Ass Bench"

sethkim — Tue, 22 Sep 2026 22:09:18 +0000

Folks, we've reached the top.

Show HN: Jev-align, a CLI to calibrate Jev to your judgement

sethkim — Sat, 19 Sep 2026 23:05:55 +0000

Article URL: https://github.com/sutro-sh/jev-align

Comments URL: https://news.ycombinator.com/item?id=49770872

Points: 3

# Comments: 0

New comment by sethkim in "The Analytical AI Handbook"

sethkim — Fri, 28 Aug 2026 20:22:51 +0000

I appreciate the classic HN sarcasm!

The Analytical AI Handbook

sethkim — Fri, 28 Aug 2026 19:01:47 +0000

Article URL: https://handbook.sutro.sh

Comments URL: https://news.ycombinator.com/item?id=49482925

Points: 49

# Comments: 2

Useful Black Boxes

sethkim — Mon, 13 Apr 2026 15:48:26 +0000

Article URL: https://sethkim.me/l/thesolutionspace/?postnum=12

Comments URL: https://news.ycombinator.com/item?id=47753761

Points: 3

# Comments: 0

New comment by sethkim in "If DSPy is so great, why isn't anyone using it?"

sethkim — Mon, 23 Mar 2026 16:44:21 +0000

This is extremely true. In fact, from what we see many/most of the problems to be solved with LLMs do not have ground-truth values; even hand-labeled data tends to be mostly subjective.

New comment by sethkim in "If DSPy is so great, why isn't anyone using it?"

sethkim — Mon, 23 Mar 2026 16:15:34 +0000

Feel free to shoot me a note at seth@sutro.sh if you want to check it out!

New comment by sethkim in "If DSPy is so great, why isn't anyone using it?"

sethkim — Mon, 23 Mar 2026 16:13:21 +0000

We build a product that's somewhat similar in spirit to DSPy, but people come to us for different reasons than the OP listed here.

1) It's slow: you first have to get acquainted with DSPY and then get hand-labeled data for prompt optimization. This can be a slow process so it's important to just label cases that are ambiguous, not obvious.

2) They know that manual prompt engineering is brittle, and want a prompt that's optimized and robust against a model they're invoking, which DSPy offers. However, it's really the optimizer (ex. GEPA) doing the heavy-lifting.

3) They don't actually want a model or prompt at all. They want a task completed, reliably, and they want that task to not regress in performance. Ideally, the task keeps improving in production.

Curious if folks in this thread feel more of these pains than the ones in the article.

New comment by sethkim in "My trick for getting consistent classification from LLMs"

sethkim — Mon, 20 Oct 2025 22:04:44 +0000

Under-discussed superpower of LLMs is open-set labeling, which I sort of consider to be inverse classification. Instead of using a static set of pre-determined labels, you're using the LLM to find the semantic clusters within a corpus of unstructured data. It feels like "data mining" in the truest sense.

New comment by sethkim in "The Future (and Present) of AI Is Synthetic Data"

sethkim — Fri, 26 Sep 2025 19:12:22 +0000

The models you called out at the beginning were all released this year. What do you think is the difference between this generation of models and previous ones?

New comment by sethkim in "The End of Moore's Law for AI? Gemini Flash Offers a Warning"

sethkim — Thu, 03 Jul 2025 18:53:03 +0000

Yes! Both Llama 3 and Gemma 3 have 128k context windows.

New comment by sethkim in "The End of Moore's Law for AI? Gemini Flash Offers a Warning"

sethkim — Thu, 03 Jul 2025 18:50:04 +0000

Yes, we're a startup! And LLM inference is a major component of what we do - more importantly, we're working on making these models accessible as analytical processing tools, so we have a strong focus on making them cost-effective at scale.

New comment by sethkim in "The End of Moore's Law for AI? Gemini Flash Offers a Warning"

sethkim — Thu, 03 Jul 2025 18:47:18 +0000

My two cents here is the classic answer - it depends. If you need general "reasoning" capabilities, I see this being a strong possibility. If you need specific, factual information baked into the weights themselves, you'll need something large enough to store that data.

I think the best of both worlds is a sufficiently capable reasoning model with access to external tools and data that can perform CPU-based lookups for information that it doesn't possess.

New comment by sethkim in "The End of Moore's Law for AI? Gemini Flash Offers a Warning"

sethkim — Thu, 03 Jul 2025 18:40:08 +0000

No doubt prices will continue to drop! We just don't think it will be anything like the orders-of-magnitude YoY improvements we're used to seeing. Consequently, developers shouldn't expect the cost of building and scaling AI applications to be anything close to "free" in the near future as many suspect.

New comment by sethkim in "The End of Moore's Law for AI? Gemini Flash Offers a Warning"

sethkim — Thu, 03 Jul 2025 18:29:54 +0000

Both great points, but more or less speak to the same root cause - customer usage patterns are becoming more of a driver for pricing than underlying technology improvements. If so, we likely have hit a "soft" floor for now on pricing. Do you not see it this way?

The End of Moore's Law for AI? Gemini Flash Offers a Warning

sethkim — Thu, 03 Jul 2025 17:34:05 +0000

Article URL: https://sutro.sh/blog/the-end-of-moore-s-law-for-ai-gemini-flash-offers-a-warning

Comments URL: https://news.ycombinator.com/item?id=44457371

Points: 113

# Comments: 75

New comment by sethkim in "Making 2.5 Flash and 2.5 Pro GA, and introducing Gemini 2.5 Flash-Lite"

sethkim — Tue, 17 Jun 2025 19:29:38 +0000

I run a batch inference/LLM data processing service and we do a lot of work around cost and performance profiling of (open-weight) models.

One odd disconnect that still exists in LLM pricing is the fact that providers charge linearly with respect to token consumption, but costs are actually quadratic with an increase in sequence length.

At this point, since a lot of models have converged around the same model architecture, inference algorithms, and hardware - the chosen costs are likely due to a historical, statistical analysis of the shape of customer requests. In other words, I'm not surprised to see costs increase as providers gather more data about real-world user consumption patterns.

New comment by sethkim in "Ask HN: Who is hiring? (June 2025)"

sethkim — Tue, 03 Jun 2025 00:00:17 +0000

Sutro.sh (fka Skysight) | Infrastructure/LLMs & Research Engineering | SF Bay Area | Full-time

We are building batch inference infrastructure and a great/user developer experience around it. We believe LLMs have not yet been meaningfully unlocked as data processing tools - we're changing that.

Our work involves interesting distributed systems and LLM research problems, newly-imagined user experiences, and a meaningful focus on mission and values.

Open Roles:

Infrastructure/LLM Engineer — https://jobs.skysight.inc/Member-of-Technical-Staff-Infrastr...

Research Engineer - https://jobs.skysight.inc/Member-of-Technical-Staff-Research...

If you're interested in applying, please send an email to jobs@sutro.sh with a resume/LinkedIn Profile. For extra priority, please include [HN] in the subject line.

New comment by sethkim in "Ask HN: Who is hiring? (May 2025)"

sethkim — Thu, 01 May 2025 22:42:32 +0000

Skysight | Infrastructure/LLMs & Research Engineering | SF Bay Area | Full-time

We are building large-scale batch inference infrastructure and a great/user developer experience around it. We believe LLMs have not yet been meaningfully unlocked as data processing tools - we're changing that.

Our work involves interesting distributed systems and LLM research problems, newly-imagined user experiences, and a meaningful focus on mission and values.

Open Roles:

Infrastructure/LLM Engineer — https://jobs.skysight.inc/Member-of-Technical-Staff-Infrastr...

Research Engineer - https://jobs.skysight.inc/Member-of-Technical-Staff-Research...

If you're interested in applying, please send an email to jobs@skysight.inc with a resume/LinkedIn Profile. For extra priority, please include [HN] in the subject line.