I'm wondering what is the overall thesis/plan here and where exactly the innovation lies? Would love if you can throw light on below,

  - If I understand, this is complete stack of a custom architecture (attention only transformers), custom quantisation format and a runtime engine all packaged together?
  - How do you differentiate / compete against LiteRT (former TensorFlowLite) and Lite RT LM? Google is heavily investing in this ecosystem because Android is where they have distribution moat. Wouldn't it be easier for me as a developer to build on top of LiteRT since it is relatively open ecosystem and I can pack large number of open models from HF directly?
  - What exact challenges you saw with TFLite, TVM etc that prompted this effort ?
  - What will be the pricing model like?

I've found many hard-core Physics textbooks to be off putting. Because the syllabus and its progression makes it seems like we are jumping from one special formula to another and there's just so much to learn! I was trying to search for a physics book that will try to cover as much ground as possible in fewer ideas and mental models. Roughly similar to Elon's idea that knowledge is a semantic tree and one should focus on trunk and big branches first.

Thomas A Moore did this experimental syllabus / series called "six ideas that shaped Physics". Each textbook takes one idea like "conservation laws constrain interactions". It's an excellent series, each book is relatively short and has amazing explanation. Especially look out for 2nd edition because later editions were probably "mainstreamed" by editors / publishers. The series is great choice for self learners. It uses non-standard notation and terminology at places to get the point across effectively so maybe not a great textbook lol.

2 standout examples for me : the first unit starts from the idea of interactions -> change in momentum -> to talk about change in kinetic energy. This motivates the idea of work done from first principles instead of directly throwing a definition "force . displacement". The unit on electromagnetic fields similarly has a beautiful discussion to motivate why one would use the ideas of curl and divergence.

Short term, we can compare this to 2-3 recent (mini) revolutions: internet, mobile, cloud. Then the answer is somewhat predictable and (somewhat sad personally). Companies owning the main distribution of intelligence (big labs) or distribution of the app/cloud layer (Google, MSFT, AWS) will make most of the money. In fact Google looks well positioned that way with owning intelligence, cloud (and even hardware, if they can get TPUs right as commercial product).

Long term view is interesting and somewhat satisfying (again, personally). We can compare this to industrial revolution, but for intelligence instead of physical labour. I hope, to borrow from Alan Kay's words, the total value generated will be more than what few big labs can capture. Though we will also see normal market dynamics of boom and bust in play. Companies building something useful, patiently will keep winning the markets. But only to get challenged by newer modes of the technology emerging.

In this long term view, the technology per se doesn't offer monopolistic profits to big labs. I think Anthropic is well aware of this and they are trying to extract as much cash from white collar work automation as they can before things are democratised. Contrary to popular opinion, they are also trying to seek a regulatory capture here by to maintain monopolistic position in the US market by scare mongering about China and open source. Its a case study how they managed to keep the good boy image of themselves while doing this.

In the end, I hope the technology emerges as electricity or combustion engine cars. Yes early pioneers (e.g. Ford) were perhaps able to make lot of money. But eventually, the technology was too important to allow one party to have monopoly and we had an abundance market which enabled jobs and money for a lot more people.

Edit, postscript : Dario, Sam and even Jensen will end up looking like the new the John D. Rockefeller's of this era. I'm personally hoping Demis Hassabis actually solves something much more important (problems in diseases, biology etc) with AI.

I'm open to the idea that market forces or reality _might_ tilt the favour in more investment in AI in education and tutoring. Think about developing economies or under developed economies of the world. The state / government has to think how should they allocate budget: on agriculture subsidies or pay teachers better or spend it on energy security in increasingly hostile multipolar world or invest it in infrastructure. There are no good answers.

Some real experience of school in my part of the world (India). My child goes to a somewhat costly private school. Still, each class has 35-40 kids. The teacher is over-worked : checking home work, prepares kids for upcoming random extra curricular activity, has to teach AI because someone suddenly thought it is important to teach it in grade 3. I know multiple teachers in that school who landed the job after doing just a 6 months course after a career break. Forget research on teaching, hardly any one of them read anything beyond curriculum. None of the teachers have enough time to give personalised attention to _any_ kid. Its a sorry state of affairs.

Personalised tutoring produced geniuses in last century but was affordable only for a wealthy few. It is my belief that AI might help democratise the idea. I know it is somewhat hard to think that a mere machine might have more patience and time to explain a concept 100 times to a student, instead of a human.

This still doesn't take away from the fact that teachers deserve to be paid more. I was a passionate for becoming a teacher, but knew I can't make enough money from it. I'm seeing a possibility in emerging markets that India / China might find it cheaper to deploy AI en masse for better educational outcomes because it will be much cheaper for the state than paying high wages (and subsequently pensions) to human teachers, however unfair it might look.

In my part of the world, "become a teacher" is often a job advised to people who are not able to find other jobs or are looking for a safe way back after career break. None of them are looking forward to engaging 5 year old with life's curiosities. To add the famous quip from WorryDream/Bret Victor : most of the teachers teaching calculus etc. have never ever used it in real life.

Working parents with STEM backgrounds likely know that schools are glorified day-cares and probability of your child having access to a life changing tutor is very low.

I tried building an edtech venture frustrated precisely with these problems. Failed, but would def do it again with AI in the mix. I'm for one rooting for this to succeed!

A closest counter path I would have expected Modular to follow was Zig or Oxide computers (I know not apples to apples comparision). Start actually attacking the problem with hindsight and lessons of 30 years of Python, build something fresh, and try to patiently win the market.

Rust is not going to win this market. The language has too much syntax friction to win over data science/AI folks and doesn't offer too much in parallel programming world. Julia, although beautiful attempt, couldn't gather enough support outside academia.

In fact, if Nvidia cuTile, Triton, Jax keep delivering, Python seems unmatched at the moment. It is likely to be in the similar position that C/C++ have been in embedded and firmware world.

I’ve seen and used airflow , spark , temporal for many systems. I’d def pick this simpler choice for 95% workloads these days.

Airflow perhaps fits better with scheduled recurring ETL workflows.

JAX seems well engineered. One would argue so was TensorFlow. But ideas behind JAX were built outside Google (autograd) so it has struck right balance with being close to idiomatic Python / Numpy.

PyTorch is where the tailwinds are, though. It is a wildly successful project which has acquired ton of code over the years. So it is little harder to figure out how something works (say torch-compile) from first principles.

The benchmarks are here: https://github.com/maximhq/bifrost/blob/main/docs/benchmarks...

Some features: • Built-in governance and routing rules • Supports over 1,000 models from different providers • MCP gateway included (HTTP, SSE, and console transport) • Out-of-the-box observability and OTel-compatible metrics

Comments URL: https://news.ycombinator.com/item?id=44820784

Points: 8

# Comments: 1

On the other hand, I think MSFT was trying to pull a classic MSFT on AI. They thought they can piggyback on top of OpenAI's hard-work and profit massively from it and are now having second thoughts, thats better too. MSFT has mostly launched meh products on top of AI.

Hope you get some nice time off and go back with vigour to Turing's Dream now...

[0] : https://cognitivemedium.com/srs-mathematics

Job seekers currently in college have no idea what is about to hit them in 3-5 years.

Take CS 231, for example, which stands as one of Stanford's most popular AI/ML courses. Think about the number of students who have taken this class from around 2015 to 2017 and have since advanced in AI. It's fair to say a good chunk of credit goes back to that course.

Instructors who break it down, showing you how straightforward it can be, guiding you through each step, are invaluable. They play a crucial role in lowering the entry barriers into the field. In the long haul, it's these newcomers, brought into AI by resources like those created by Karpathy, who will drive some of the most significant breakthroughs. For instance, his "Hacker's Guide to Neural Networks," now almost a decade old, provided me with one of the clearest 'aha' moments in understanding back-propagation.