I don't think technology will fizzle out in either scenario. In the 'dark age' highly competitive scenario, you will still have those who 'understand what is produced' seeking progress and advancement. They will cohort with whoever/whichever organization is producing the best, most validated results.

I don't think knowledge would be become alien to human understanding, it would just become highly concentrated again as it was in the past. In the past, only the wealthy and noble could afford to spend time on education.

And usually, those who went deep into any field and provided contributions to science were those who had time to devote entire lives to the study or pursuit of knowledge. We still have these issues today, and these issues will always be present.

There is a supply chain of economy, and we provide different rewards based on the value of what is produced. The jobs never go away, the desire for prestige, greed, etc., never goes away. It's why economies exist, and it's why everyone is considering the value of LLMs, and their place in economies, and academies (presumably which exist to advance human understanding/appreciation, but also to prepare younger generations for future work).

The dramatic solution can never be: 'we should all just want to do it this way because it felt better to me'. That never seems to work, from what I've seen. So there is a lot of public discourse, arguing, to determine if 'doing things the old way' is perhaps more better. And if not, maybe we just shed a tear and move on (to look for a better fit).

Social creatures that share a common space, with finite time that nobody wants to waste.

It's the same reason you see 'no AI' repo labels: only by forcing some standard on the community can the 'old way of collaborating' continue to exist.

There will for sure be new ways of collaborating, and walled off gardens like you suggest, where communication stops and everyone just shares some final product.

But, in some cases the product must be understood (or demonstrated) before value can be derived, and so collaboration will still exist.

Desire, and the 'shape' of collaboration has changed greatly though. Why collaboration has changed, is a completely separate issue related to economy and how each of us spends our time.

And then once it's solved by AI: do you want to churn through the proof to understand it? What if it takes you 5 years to churn through the proof, and it turns out someone made a mistake with AI (I have no idea what would be realistic amounts of time, but the idea is still worth considering).

The issue is very complex, and you will always be hungry.

You ended your first post with: "An LLM encodes the normative regularities of our linguistic/conceptual games into its weights, but it cannot be a real participant in our games because it is incapable of staking durable commitments or changing its dispositions over time."

And so if there is an agent, a thinking loop, that can recursively adjust over time...how does your statement hold up? I'm particularly talking about 'staking durable commitments or changing its dispositions over time'. Recursive self improvement is all about these 2 things, especially when it's the 'outer thinking loop' doing the improvement.

But you're right, Mathematics exists as a medium within Language. But then Language is just a transfer of ideas from one mind to another.

This principle is the heart of what the article is talking about: a transfer of logical mathematical ideas passed from a human to another. And the sense of 'shared understanding' that comes with it. It has nothing to do with keeping score.

It's about getting a 'sense' for why someone thought or rationalized an idea in order to connect some other idea. You get an 'ah-ha' moment, and some of those moments are gone, or a different path is taken to reach the moments now. That is what the 'weeping' is about (not that I agree with the article. I think anyone can spend however they want. Math doesn't have to change, it's that people want it to, apparently, because it's 'easier' to do something different, or for some other reason I don't know about).

I think you are also suggesting Mathematics and Language are the same thing.

Both depend on Logic, which is where "deontic scorekeeping" comes in. You can only establish something (score) is true and believable by 'keeping score' and having some level of understanding about the language or math you are reading.

The article is mostly referring to the connection that comes from shared understanding between humans, particularly mathematicians, as they build new shared understanding, and realize there are limits to what they can know and 'leave behind' logic for others to pick up and carry on.

That connection has been severed, or strained a bit, but I'm sure new ones will be created. It just might be of the same flavor that mathematicians prefer, therefore they weep.

And if I understand "deontic scorekeeping" correctly, these forums are a better representation of the concept. You will lend some amount of belief to what is said/read based on what you know.

'The sun will rise tomorrow' is a pretty weak prediction, but we don't truly know that the sun will rise tomorrow. However, we can be pretty sure it will based on probability, history, likelihood, from our own past observations. You wouldn't need much convincing to believe the prediction.

'The sun will not rise tomorrow' is a much stronger prediction, because it's a 'wilder' less believable claim with very huge impact. Nobody would believe the statement without proof.

A hypothesis, is a testable statement about a predicted outcome. You could hypothesize something like: 'The Earth's core will stop due to XYZ event, which will stop Earth's rotation. The sun will not rise tomorrow.' Then you go about designing experiments (or you observe and show something that is measurable, like some asteroid on a path close to Earth's) to prove the statement.

From the article: 'Charles Darwin posited that some species from the genus Saxifraga—a group of flowering plants typically found in alpine environments—may be carnivorous because of their sticky glandular hairs, which can trap insects'.

'Posited' is to suggest, assume, or put forward an idea, theory, or fact as a foundational basis for an argument or investigation. 'Posited' isn't quite as strong as the word prediction, it's more like a sensible suggestion for further study. Darwin had observed carnivorous behavior in other plants with sticky hairs, and suggested the behavior may be observed in another plant as well.

He didn't make a strong claim like: 'The genus Saxifraga plants are carnivorous because they have sticky hairs that trap insects.' This sort of claim suggests that plants with sticky hairs always trap insects for nutrients, and frankly, he couldn't know that's true without observing every plant with sticky hairs.

What if there are plants with sticky hairs that use the stickiness as a defensive measure instead?

Should the article have used a different word than 'predicted'? Probably, but it wouldn't attract as many viewers, and 'predicted' sort of lends Darwin deserved credibility.

The math problems AI helped solve were problems identified by humans in the past (lighthouses, for the field of math). They're problems meant to guide the field: working towards solving the problems brings new ideas to light, and possibly steers us into new areas we wouldn't have considered in the past. More is considered, and possibly more is discovered along the way. (It's very difficult to validate that statement, and so there's plenty of room for argument, and is the heart of the discussion)

Using AI to take a shortcut to the lighthouse creates a survivorship bias. All attention is spent on proving solved problems: less is considered, and certainly less is understood.

Academia was set up for the old journey to the lighthouse, and if we don't consider carefully the new journey to the lighthouse, we'll lose all ability to create new lighthouses (or steer ships).

He seems to suggest it's more valuable for humanity to integrate AI into the academic pipeline, versus spending all attention on solving existing open problems.

Facts are established truths intelligent beings use to build knowledge and increase accuracy of prediction: and facts are acquired through observation.

If Logos was there in the beginning: then so are intelligence, meaning, observation, and fact.

We're a part of the universe, and we ask the question: what came first? A universe asks itself: what came first?

These words only carry meaning because you are an intelligent observer.

Eco: house, household, or dwelling place.

The -logy suffix is 'science'. So ecology is: the scientific study of how living organisms interact with one another and with their physical surroundings.

The -nomy suffix is a field or system of something.

Economy is the ecology of finance. Or, Economy is a field of science that predicts financial systems. Any scientific field helps you make predictions, once you know the material. But useful predictions require useful measurement.

I do like your gameplay mechanic idea, but others may not. A quick filter may not add "soul" for you, but it might for others. It's about emotions and immersion, but it's all obviously illusion.

The opposite, where everything is very clearly manicured, created, is so far from reality that the immersion is more difficult to achieve, or is quickly lost. You never stray far from the fact that you're playing a city game with very rigid rules.

I think this is what the author means by "Soul": objects with details to show something lived, was paid attention to, has story, depth, and scars to prove it.

Also, Hackernews tries to hold itself to a certain standard. Reading an article about an AI Engineer who gives an AI agent access to a tool, with clear directive and access to use the tool, who is then later confused when the agent uses the tool in a slightly different way, suggests the article is below Hackernews standards.

To answer your question though: yes you absolutely could fight spam with AI on your side.

Should we go down that path as a society, just for amusement? Essentially saying: yes, someone should pay for AI service to combat AI spam, just so we can continue to use email.

Everyone in the article seems desperate for attention and engagement, and it makes some readers uncomfortable: because of the awkwardness, and because this guy is spreading a 'bad idea' that probably shouldn't be replicated.

Imagine if everyone starts spamming emails? The researchers are amused this time, but I guarantee they wouldn't be if their inbox filled up with agent emails.

Not to mention the way the agents reached out for help is probably a bad idea: asking someone with 30 years of experience to apply all their knowledge to your small, specific problem in another knowledge domain, with no other detail or context puts all the mental burden on the unsolicited email recipient.

It's not either or, it's all of the above: individuals do what feels good and if you're a great party thrower because it energizes you, you'll probably spread that influence some. And then possibly, the group reinforces itself by continuing to throw and attend parties.

At the same time, leisure time has changed: women work more, dads spend more time with kids, and the home offers a lot more easy entertainment that fits in with modern schedules. There seems to be less opportunities for parties with newer generations.

But of course, society seems to shun the party thrower because we're all busy. And so if nobody shows and the party thrower burns out, there's no influence spread. People dislike parties more and the idea reinforces again.

A mix of any of the above goes into our decisions about how to spend time.

You put the customer first, that's first principles thinking. You adjust the way you spend your time, according to your customer's needs.