Even if OAI had zero data from Buckmaster's sessions, this is in very poor taste and highly unethical. You are front running a researcher just to be able to say you did it first? Tao is right - OAI is treating math results like oil. This is the like Exxon getting a whiff of a massive oil field and racing to the punch by deploying their full crew.

"deidentified data" isn't much to go by. Say I prompted the internal model this way - "Hey there's a solution to a unsolved problem X. The solution uses a less known Method Y so don't bother wasting time with the usual methods. Take papers A, B and C as references. Oh btw, here's the last year's worth of data of all prompt sessions that mention this problem. Pay special attention to the ones that mention Method Y and sub-keywords Z,W".

This is obviously all speculation but the timing is very suspect. If OAI actually did this (and I suspect whatever they did is pretty much close to this), I think it is highly unethical.

Try to think of yourself as a professor who's trying to come up with a problem statement worth solving. The AI is your "lab" that will help you run experiments.

Out of curiosity, are you one of the people who has access to the model? If yes, could you write about your experimental setup in more detail?

I'm not that old but have been here long enough that I remember when GPT-3 was considered too dangerous to release. Now you have models 10x as good, 1/10th the size and run on 8GB VRAM.

I guess this is the crux of the debate. All the claims are comparing models that are available freely with a model that is available only to limited customers (Mythos). The problem here is with the phrase "better model". Better how? Is it trained specifically on cybersecurity? Is it simply a large model with a higher token/thinking budget? Is it a better harness/scaffold? Is it simply a better prompt?

I don't doubt that some models are stronger that other models (a Gemini Pro or a Claude Opus has more parameters, higher context sizes and probably trained for longer and on more data than their smaller counterparts (Flash and Sonnet respectively).

Unless we know the exact experimental setup (which in this case is impossible because Mythos is completely closed off and not even accessible via API), all of this is hand wavy. Anthropic is definitely not going to reveal their setup because whether or not there is any secret sauce, there is more value to letting people's imaginations fly and the marketing machine work. Anthropic must be jumping with joy at all the free publicity they are getting.

Unless Anthropic makes it known exactly what model + harness/scaffolding + prompt + other engineering they did, these comparisons are pointless. Given the AI labs' general rate of doomsday predictions, who really knows?

Tbf I think the golden days of being a software dev are over even if the AI were to stagnate and never improve. The spectre of AGI is enough for higher ups to demand more output which will in turn require more hours to be put in by devs. A project that required 2 months will now be allotted 3 weeks because "Agentic coding increases productivity".

This exactly. I am neither a boomer nor a doomer. It has helped a lot both at work and in accelerating my personal projects. But now that the C-suite and middle management has jumped on the agentic bandwagon, I'm unsure where this will go and what casualties ensue. At the very least, in the short term there's going to be a lot of "Now that we have agents, this project should be achievable in half the time".

Using computers to aid in designing is not specific to Kanchipuram saris. While I realize people always approach it from the POV of saving a dying art, I'm unsure if K.saris can really fall under that umbrella. Clearly the demand is there and the issues here arise due to inefficient and possibly corrupt market practices rather than the art itself dying. A lot of space was used to explain the lopsided economics on the supply side but there's not enough attention paid to the demand side and the marketplace dynamics.

In general, hedonic adaption ends either with internal retrospection (shifting from pleasure to purpose) or an external disruption. In America's case, the former is extremely unlikely IMHO - the American people will not put their money where their mouth is because they enjoy the wealth generated this way. It will be upto external disruptors to check on Uncle Sam's endless thirst.