I think the bet would have to be that a US Open Weight company either: 1. Gets a lot of money from Jenson who views them as a counterbalance to the big labs in his ecosystem and a way to generate leverage (the same way he is positioning neoclouds-- it also could be synergistic with neoclouds who could offer the model serving endpoints) 2. Can fast follow the same way Mistral does (which, honestly, seems like just distilling the Chinese model, which distills the US lab but is pretty innovative on a whole lot of architecture both in training and serving land.) 3. AND figure out some (maybe not super lucrative but lucrative enough) sort of business model, as well. There are lots of possible business models, so I will be curious how this whole space evolves.

The model could also be more flexible for non-coding use-cases (they show the results for reasoning being strong) so maybe the argument is to use it for non-coding use-cases to drive relatively deterministic conclusions for non-coding agents (they have also done some determinism work on kernels, which could be useful in pulling on that thread of deterministic models that are fine-tuned for everything that is not writing code.)

That said, I'm not sure how much all the work they have done actually synergizes or if the market size (at least in the short to medium term) is big enough for a huge outcome from the company's current valuation with those bets as the enterprise agent estate is taking a while to evolve. Hence companies like Anthropic and OpenAI are throwing tons of consulting money at the problem.

That said, the fine-tuning API + open weight model at least is a semblance of a viable business that could work so I will be curious about it. I'm not sure the synergy is fully there (why is someone with an open weights model privelaged to fine-tune it better if it's just QLora or Lora) but let's see!

Part 1 is the theoretical foundation. It walks through a modern algebraic formalism showing that FlashAttention is an associative operation, which lets you treat it as a regular reduction on the GPU and apply all the same scheduling optimizations. Some recent MLSys and CVPR (ELSA) papers lean on this framing, and I find it much more powerful than the original.

This framework is particularly useful in ML compilers, where you should implement general optimizations applicable to many operations rather than writing specialized kernels. This article shows that attention actually belongs to a large family of "secretly-associative" operations, walks through a handful of examples, and links a few concepts from abstract algebra that let you identify whether an operation is secretly associative.

Overview:

- Safe softmax, Welford's variance, and FlashAttention are the same secretly-associative operation

- The twisted monoid (transport of structure), why the max-rescale coupling doesn't break associativity

- The qk_scale = log2(e)/√D like in FA-2 derived from scratch

- Numerical analysis: overflow bounds, error limits, and why tiling never amplifies error

- Bird's 3rd Homomorphism Theorem as a test for whether any loop is secretly associativ

Comments URL: https://news.ycombinator.com/item?id=48920785

Points: 8

# Comments: 1

Do you think this would scale to larger transformer models with more parameters per layer?

How would this work with MOE models or sparse models?

Comments URL: https://news.ycombinator.com/item?id=48022994

Points: 10

# Comments: 0

Comments URL: https://news.ycombinator.com/item?id=47965387

Points: 1

# Comments: 0

Comments URL: https://news.ycombinator.com/item?id=45997614

Points: 7

# Comments: 1

The TLDR/key (from my understanding) is that verifying N tokens can be faster than generating N tokens.

Feels like two types of companies raised money: - Companies trying to couple the cloud with a programming language. - More recently, companies trying to couple GPUs with a programming language/alternative to CUDA.

Will be curious how this generation goes.

I don’t see those deals on Uber Eats so it feels like the real value of Uber One is for heavy Uber Eats users.

PS. Worth going through the cancellation flow when you are up for renewal as they will probably offer you 50% off Uber One.

VCs went a bit ham on this category when Auth0 got bought. I sense that the general thought process was: Auth0 multi-billion dollar company -> Auth0 will become crappy under Okta > replace Auth0 and build a bit company.

Dbt introduced a language for managing these “metrics” at scale including the ability to use variables and more complex templates (Jinja.)

Then you do dbt run (https://docs.getdbt.com/reference/commands/run) and kapow the metric is populated in your database.

More broadly dbt did two other things: 1. It pushed the paradigm from ETL to ELT (so stick all the data in your warehouse and then transform it rather than transform it at extraction time.) 2. It created the concept of an “analytics engineer” (previously know as guy who knows SQL or business analyst.)

I would be curious to learn more about the rationale to sell the business as I understand the strategic value to Amplitude. Interestingly, these next-generation digital adoption platforms have generally been pretty challenged.

How does this compare to Lance (https://lancedb.github.io/lance/)?

What do you think the key applied use case for Vortex is?