It's a race metaphor not a football metaphor.

In foot/cycling races there's often a pack leader, that leader is often not the winner of the race, all they're doing is taking the brunt of the air resistance while everyone else slipstreams behind. For a casual observer it seems that the pack leader will win, but everyone knows that it's gonna be someone that paced themselves that's going to overtake the first spot at the tail end of the race.

What's the point of leading the race for 90% of it, if they're gonna slip on their own sweat and fall down by the end? In non metaphorical terms, what's the point of spending billions of dollars rushing to get the best AI tech at all costs, when the competition can distil your progress and catch up in 6-12 months while only spending 1% of what you spent.

Even in the aspect the article cares about, commercialization, the US is starting to lose marketshare, I've seen people move from cc/codex plans to use glm/opencode plans due to the recent squeeze the US companies put on plan usage, the US companies are screwed if that sticks, not everyone needs the bleeding edge models, they just want to pay $20/month and have the models be decently capable.

To me it stands to reason that a model that has only seen a limited amount of smut, hate speech, etc. can't just start writing that stuff at the same level just because it not longer refuses to do it.

The reason uncensored models are popular is because the uncensored models treat the user as an adult, nobody wants to ask the model some question and have it refuse because it deemed the situation too dangerous or whatever. Example being if you're using a gemma model on a plane or a place without internet and ask for medical advice and it refuses to answer because it insists on you seeking professional medical assistance.

I've never heard of ANY speculative decoding that wasn't lossless. If it was lossy it'd be called something else.

This page is just a port of DFLASH to gguf format, it only implements greedy decoding like you said so the outputs will be inferior, but not inferior to greedy decoding on the original model. Tho that's just a matter of implementing temperature, top_k, etc.

When you already know the tokens ahead of time you can calculate the probabilities of all tokens batched together, incurring significant bandwidth savings. This won't work if you're already compute bound so people with macs/etc. won't get as much benefits from this.

I don't watch sports, but my father watches soccer. He really only cares about 1 team and the national games from our home country. He was spending over $100/month to be able to watch the games, and they werent even in his native language. Now he pays $80/year for a pirate IPTV service and not only can he watch the games anywhere he wants, he also gets native language commentary for the games, national tv channels like news, etc.

When pirates can charge you money and offer a superior service, it absolutely is a service problem. You can claim that the realities of licensing and whatnot don't allow official channels to provide the best service they can, but that's not true in this case. When the same provider is splitting game broadcast from one team into different packages you know they're just trying to extract the most amount of money possible.

IDK the deal with scanlator sites nowadays, but I assume the official sites can provide more timely translations for manga since they can access the source material before anyone has seen it. I know most popular manga gets translated within hours of release, but if you're following some more niche stuff it can be several days. I also know a lot of scanlators have patreon pages so it's not like the demand from paying customers for translated media isn't there.

They introduced a 1M context model semi-transparently without realizing the effects it would have, then refused to "make it right' to the customer which is a trait most people expect from a business when they spend money on it, specially in the US, and specially when the money spent is often in the thousands of dollars.

Unless anthropic has some secret sauce, I refuse to believe that their models perform anywhere near the same on >300k context sizes than they do on 100k. People don't realize but even a small drop in success rate becomes very noticeable if you're used to have near 100%, i.e. 99% -> 95% is more noticeable than 55% -> 50%.

I got my first claude sub last month (it expires in 4 days) and I've used it on some bigish projects with opencode, it went from compacting after 5-10 questions to just expanding the context window, I personally notice it deteriorating somewhere between 200-300k tokens and I either just fork a previous context or start a new one after that because at that size even compacting seems to generate subpar summaries. It currently no longer works with opencode so I can't attest to how it well it worked the past week or so.

If the 1M model introduction is at fault for this mass user perception that the models are getting worse, then it's anthropics fault for introducing confusion into the ecosystem. Even if there was zero problems introduced and the 1M model was perfect, if your response when the users complain is to blame it on the user, then don't expect the user will be happy. Nobody wants to hear "you're holding it wrong", but it seems that anthropic is trying to be apple of LLMs in all the wrong ways as well.

Instead of asking the model: "Here's this codebase, report any vulnerability." you ask. "Here's this codebase, report any vulnerability in module\main.c".

The model can still explore references and other files inside the codebase, but you start over a new context/session for each file in the codebase.

I upload edited gameplay vods of twitch streams on youtube, and use whisper-large-v3 to provide subtitles for accessibility reasons (youtube's own auto-subtitles suck, tho they've been getting better).

My checklist for a good ASR model for my use case is:

1. Have timestamp support.

2. Support overlapping speakers.

3. Accurate transcripts that don't coalesce half words/interrupted sentences.

4. Support non verbal stuff like [coughs], [groans], [laughs], [sighs], etc.

5. Allow context injection of non-trivial sizes (10k+ words)

1 is obvious because without it we can't have subtitles. Force alignment fails too often.

2 is crucial for real world scenarios because in the real world people talk over each other all the time, in my case it's a streamer talking over gameplay audio, or when the streamer has guests over. When 2 people speak the transcript either ignores one of them, or in the worst case, both of them.

3 and 4 are an accessibility thing, if you're deaf or hard of hearing having a more literal transcript of what's being said conveys better how the speaker is speaking. If all subtitles are properly "spell-checked" then it's clear your model is overfit to the benchmarks.

5 Is not a requirement per se, but more of a nice to have. In my use cause the streamer is often reading stream chat so feeding the model the list of users that recently talked, recent chat messages, text on screen, etc. Would make for more accurate transcripts.

I've tried many models, and the closest that fulfill my needs are LLM style models on top of forced alignment. It's too slow, so I've been sticky with whisper because with whisperx I can get a transcript in 5 minutes with just a single command.

One thing all these models do (including whisper) is just omit full sentences, it's the worst thing a model can do.

https://i.imgur.com/c0a3vUy.png

DLSS is also AI and people like it.

People don't like framegen because the manufacturers are not being honest about it and using it for deceptive hype marketing. Anyone with a brain knows that it introduces latency and is only useful if you're already 40+ FPS, we also know that companies will use it to pad benchmarks. NVIDIA themselves said that the 5070 had 4090 performance because it supports framegen.