This level of targeting is really only something that nation state actors can pull off, who are also protected from actual law enforcement prosecution. Unless attackers are really in North Korea, they would 100% be caught within just few days.

Since that is clearly not the case, that is just as much Anthropic's problem as it is Cursor's problem. There either needs to be a technical change so that extensions are signed with keys that Anthropic controls, or you should take the branding off from the Cursor extensions.

If you cranked up turns and only used fable level model for every interaction, you would get way way less jank. Plus when you have essentially unlimited tokens, you can always do extra AI review passes to reduce jank.

I think for us, outside of Anthropic, it is hard to imagine the scale of AI agents that they are running internally . We are talking about probably 10s of millions $ worth of spend monthly if not more.

We are talking billion dollars+ worth in a single address. So if either chainskey security is compromised in a way that it is conceivable to brute force it, there will definitely be plenty of targets.

On that note, I actually had an overall harness (for experimenting) that was essentially like this: "for any task, instead of answering question directly, write a program to answer the question instead. test and verify the program before giving the answer".

It actually worked incredibly well on all "gotcha" LLM questions like math or counting letters in words and all sorts of stuff.

Of course it was ridiculously slow and very expensive but it was a proof of concept that it can actually be much more accurate on every task if you are willing to spend an infinite amount of money.

It will probably do a pretty good job if you ask it that way (it will also burn a shit ton of tokens, but hey, that is part of the fun).

On that note, I actually had an overall harness (for experimenting) that was essentially like this: "for any task, instead of answering question directly, write a program to answer the question instead. test and verify the program before giving the answer".

It actually worked incredibly well on all "gotcha" LLM questions like math or counting letters in words and all sorts of stuff.

Of course it was ridiculously slow and very expensive but it was a proof of concept that it can actually be much more accurate on every task if you are willing to spend infinite amount of money.

Only apps that get flagged by the AI reviewer then have to go through a separate, slower human review process.

There would be no need to deliver new mathematical insights by solving problems. You would just have a magical math problem solving machine and that’s it.

Turns out it wasn’t actually Anthropic and just a researcher with a single Anthropic guy friend working on it .

Wild times

But within next 10 years as costs drop significantly and even more improvements are made, yes it is very likely that almost every single existing math problem will get a serious AI cracking done on it

Imagine if OpenAI opened up a high frequency trading arm and suddenly stole all the prompts and research that other HfT firms are doing through OpenAI tools and start making bank based on that . Wouldn’t that be straight up insane?

Reading Sam Altman's and Sebastien's tweets reads like something written by people who know they did dirty shit and are willing to cross any lines to "win". https://x.com/sama/status/2097385167002415140

OpenAI's leadership just can't help to continue to disappoint everyone with their lack of ethics or integrity.

The reality is clear though. The chances of AI models overtaking majority of mathematics within next 10 years is becoming very high. Especially if it becomes cheaper to run these models.

As math formalizations improve, AI can have faster progress in math, compared to even computer science or software engineering.

It is simultaneously the best and the worst time to be a mathematician right now.

Easiest example would be groceries. If you live in a crazy expensive place like Manhattan, you could easily spend 4-5x more on groceries compared to a city like Warsaw. But it's not like you are getting better produce or products in Manhattan, you are getting same stuff, just paying way more.

And this is true for other day to day stuff too like Housing, Healthcare, Schools, etc.

Of course rational thing to do would be to take the money and move to a less expensive city, but with the same logic, anyone who has 1m net-worth could liquidate their assets, move to a different country, retire and live a proper wealthy lifestyle.

But we don't see that happening all that much at all. Mostly what happens is that people are choosing to live middle class lifestyles in most expensive areas that they can afford.

But people's perception of their own wealth depends way more on relative value of their wealth compared to cost of living. If you live in a city where some shitty 2 bedroom apartments cost 1m$+, you aren't going to feel all that rich even if you have 10 million dollar net worth.

Especially if you are doing "remote" development through SSH. If you are doing stuff where you still have to write some parts of the code manually or you have to fix few things here and there that the AI outputs, you still need a real editor.