Data science, and then ofc DL being done through python just when python 3 was kinda usable (around 3.3/3.4) was a struck of luck timing-wise.

Cubase, etc. have no such link that I know how, but there was still the strong hacker culture around atari and to a lesser degree amiga (vs PC), when PC was just not usable for anything low latency in mid 90ies.

I cloned the repo of said library, gave it claude and asked it to write a new technical report in math notation, but with annotation with link to the code so that I can pick up the details. It basically one shotted the full report and that helped me re-implement it in "pure python + numpy", "manually".

For example, ableton was famousley co-created by the members of the monolake, a pioneer of minimalist techno in the 90ies. Some history there: https://www.roberthenke.com/interviews/ableton.html

The rest will be from "python float" (e.g. not from numpy) to C, which gives you already 2 to 3 order of magnitude difference, and then another 2 to 3 from plan C to optimized SIMD.

See e.g. https://github.com/Avafly/optimize-gemm for how you can get 2 to 3 order of magnitude just from C.

I started learning about GPU and CUDA from this book recently, and I agree the writing is confusing, and code examples have errors. However, it is still a nice reference about many types of algorithms for heterogeneous memory devices, it helped me understand better some patterns for CPUs.

It is guaranteed failure mode of large orgs. Curious to hear about more references on how to fight this at an organization level, besides the one given in the OT.

So increasing individual output by itself is not enough to affect the argument. It could, if you also reduce the size of people needed for a project, where people are everyone included in the project, not just SWE. But there are strong forces in large orgs to pull toward larger project sizes: budgeting overhead and other similar large orgs optimize for legibility kind of arguments.

IMO the only way this will change is when new companies will challenge existing big guys. I think AI will help achieve this (e.g. agentic e-commerce challenging the existing players), but it will take time.

I did some ML in mid 2000s, and it was a PITA to reuse other people code (when available at all). You had some well known libraries for SVM, for HMM you had to use HTK that had a weird license, and otherwise looking at experiments required you to reimplement stuff yourself.

Late 2000s had a lot of practical innovation that democratized ML: theano and then tf/keras/pytorch for DL, scikit learn for ML, etc. That ended up being important because you need a lot of tricks to make this work on top of "textbook" implementation. E.g. if you implement EM algo for GMM, you need to do it in the log space to avoid underflow, DL as well (gorot and co initialization, etc.).

One of the key value is that it forces some thinking about what is the task you want to solve in the first place. In many cases, it is difficult if not impossible to do it, which implies the underlying product should not be built at all. But nobody wants to hear that.

Doing eval only makes sense if making the product better impacts something the business cares about, which is very difficult to do in practice.

Concretely, managing 12 ICs on a well defined platform team w/ a single PM is much easier than managing 6 people working across 6 businesses, as is more common when managing a team of data scientists.

My sense is that unless actively managed against, any org big enough to have a financial department and financial planning will work under assumption of fungibility.

Concretely, it made distributing OSS VST plugins a pain. Especially for Linux which generally will want to build their packages.

But MCP has at least 2 advantages over cli tools

- Tool calling LLM combined w/ structured output is easier to implement as MCP than CLI for complex interactions IMO.

- It is more natural to hold state between tool calls in an MCP server than with a CLI.

When I read the OT, I initially wondered if I indeed bought into the hype. But then I realized that the small demo I built recently to learn about MCP (https://github.com/cournape/text2synth) would have been more difficult to build as a cli. And I think the demo is representative of neat usages of MCP.