I imagine they are panicking a little with LLMs reducing the reliance on external companies for software support, so they are really hoping to shift it to on-prem AI, which is where the bias in his statement comes from.

But this only works for frontier models. If MiniMax catches up to Opus 5.5 in 6 months, no one would notice.

I wish this will be true, but right now, every threshold we establish tend to be crossed over in a blink of an eye.

I noticed there's no Google Trends equivalent for Reddit, so I built it! Unlike search, which shows what people are looking up, Reddit shows what they're talking about, so I feel it complements Google Trends as another interesting data source. It also has the benefit of being able to narrow things down to a single subreddit.

I've normalized trends by unique posters per day instead of raw mentions. It's not perfect, but without it a single account (e.g. AutoModerator on one subreddit) can make it look like there's a trend when there isn't, especially for less common words.

I built a few features I haven't seen elsewhere, and ended up using them a lot:

Sentiment - the weekly net sentiment of posts and comments mentioning the word (not sentiment toward the word itself), using cardiffnlp's twitter-roberta model. For example, you can see that mentions of AI have been getting more negative, and ChatGPT had one very positive week at launch and has been mostly negative since.

Related terms - words that show up around your word more than they do elsewhere. It works better than I expected, even for rare words. A small SaaS I follow has only ~100 mentions in all of Reddit, and it still surfaced related companies.

Peak explanation - finds the post that best explains a sudden spike. In an AI-graded sample, at least one of the top 3 posts clearly explained the spike 92% of the time. It also shows at a glance why a term tends to spike. For Cloudflare and AWS, it's usually an outage.

The homepage also has an archive of Reddit's trending terms for every month back to 2005, and there's a small word chain game hidden in there too.

You can only search single words for now, because Reddit is huge (31B+ posts and comments). That turned out to be a blessing. It keeps the vocabulary to ~16M terms (anything mentioned in 100+ posts or comments), which is small enough to precompute, and that's what made sentiment, related terms and peak explanation possible. The whole site serves from a single 420 GB precomputed release from an old mac mini (I hope it won't crash!), sentiment precompute ran at ~11k docs/s on one RTX 5090, so it took a few weeks to calculate.

One caveat: it looks like Reddit is closing API access, and it's hard to get even for research and education, so the data may go stale soon. Enjoy it while it lasts.

P.S. I wanted to mark it as "Show HN:", but for some reason HN doesn't allow that prefix for new accounts, but does allow other titles.

Comments URL: https://news.ycombinator.com/item?id=50145913

Points: 7

# Comments: 1