In drug property prediction domain, tabular foundation models coupled with another foundation model for molecules are pretty close to being the state of art.

Btw the article makes heavy use of AI or is written in that way A lot of unnecessary dramatic flair that gets very tiring

Right now that entirely depends on the whims of a few people. We can try to make it not be that but I don't have much hope in that area. See climate change

> As a software engineer, I myself have only recently recovered from it

Can I ask you how you came here? Right now I am getting closer to "purpose death" as you call it. Software engineering is one of the few things I am good at and the only thing that I can rely on to put food on my table. Even if I end up being someone who gives minor suggestions to the AI, it always feels like C level execs can't wait to get rid of me

It's really hard to definitely prove it's not doing it right? Hopefully the model does not do anything like this during training because its too much work

> Well... no, it won't be nothing - it will be something. And I'm all ears for a plausible explanation, so fire away.

I'm just stating the null hypothesis that it's nothing. Especially since the agents were talking in clear english before and exchanging ideas

This reminds me of the plot of Hot Fuzz where the officer comes up with a grand narrative of what's was happening but the truth was such a mundane simple thing.

Short of OpenAI, or the agents themselves, telling the truth, we have no way of knowing what's real so let's not get carried away by grand narratives

Atleast with a well built library you know the contours and how it fits into your larger system

This seems to be what AI these days seems almost super humanly good at. See coding or math I guess.

But it does beg the question, why would a programmer using AI as a tool be worse than a programmer building the harness and environment and asking AI to go hogwild? The latter is definitely faster but if it's the former, atleast I will have an understanding how the system works. Weather that is valuable is an open question as far as I am concerned

I tried to modify the embedding output of bert to make it generate box embeddings instead of point ones. At the time I had access to university provided A100 gpus but even with all that a training run took half a day. Models these days I don't think I can train it in any reasonable time with that much compute.

https://patwalters.github.io/Response-to-Peter-Kenny/

> (4a) revert to traditional methods but keep the veneer of using ML to save face

I haven't worked in the industry side of things but in academia everyone kind of agrees that gradient boosting trees are some of the best models to do these things.

I am not familiar with chineese model companies as much as I am with US based ones so I don't have much to say beyond that the CEO is incentivced to pump up those numbers.

> By limiting the number of tokens you use per month? per week, per hour? And by limiting the inference time compute dedicated to each turn in each session.

If this was so simple I don't know why GitHub copilot went to token based billing at my company.

> This does not need to be solved, but rather only quantified. Innovation is needed to be able to reasonably bound this variance for a reasonable subset of tasks

It's much better to make a business case for them after finding this bound right? Currently I can't use copilot for anything serious since I cannot predict how many credits one request is going to consume.

Your point about non frontier tasks using less tokens makes sense. As you said, let's see if it holds up

Unless they can massively scale down training and inference cost or implement AGI I don't know what their plan is. Just provide a subsidized plan for the next 10 or 20 years? Their costs are directly proportional to the amount of tokens the LLM produces. How is a monthly subscription plan supposed to account for such costs?