I really don't get people who see this kind of thing as empowering because in the end your (now strictly necessary) appeal with lawyers or AI to get a more fair deal just becomes a new tax on your time/money; you are worse off than before. A good capitalist will notice these dynamics, and invest in AI once it's as required for life as healthcare is, and then work on driving up the costs of AI. Big win for someone but not the downtrodden.

Could be the other way around, but I think marketing-speak is taking cues here from legal-ese and especially the US supreme court, where it's frequently used by the justices. They love to talk about "ethical calculus" and the "calculus of stare decisis" as if they were following any rigorous process or believed in precedent if it's not convenient. New translation from original Latin: "we do what we want and do not intend to explain". Calculus, huh? Show your work and point to a real procedure or STFU

In the dating scenario what's really absurd and disgusting isn't actually the artificiality of toys.. it's the ritualistic aspect of the unnecessary preamble, because you could skip straight to tea and talk if that is the point. We write messages from bullet points, ask AI to pad them out uselessly with "professional" sounding fluff, and then on the other side someone is summarizing them back to bullet points? That's insane even if it was lossless, just normalize and promote simple communications. Similarly if an AI review was any value-add for AI PR's, it can be bolted on to the code-gen phase. If editors/reviewers have value in book publishing, they should read the books and opine and do the gate-keeping we supposedly need them for instead of telling authors to bring their own audience, etc etc. I think maybe the focus on rituals, optics, and posturing is a big part of what really makes individual people or whole professions obsolete

This reminds me of an awesome bit by Žižek where he describes an ultra-modern approach to dating. She brings the vibrator, he brings the synthetic sleeve, and after all the buzzing begins and the simulacra are getting on well, the humans sigh in relief. Now that this is out of the way they can just have a tea and a chat.

It's clearly ridiculous, yet at the point where papers or PRs are written by robots, reviewed by robots, for eventual usage/consumption/summary by yet more robots, it becomes very relevant. At some point one must ask, what is it all for, and should we maybe just skip some of these steps or revisit some assumptions about what we're trying to accomplish

> Despite his own moral lapses, Franklin saw himself as uniquely qualified to instruct Americans in morality. He tried to influence American moral life through the construction of a printing network based on a chain of partnerships from the Carolinas to New England. He thereby invented the first newspaper chain. https://en.wikipedia.org/wiki/Benjamin_Franklin#Newspaperman

To be clear Franklin's obviously a complicated historical figure, a pretty awesome guy overall, and I do like American pragmatism generally. But it matters a lot which part of the guy you'd like to hold up for admiration, and elevating a preachy hypocrite that was an early innovator in monopolies and methods of controlling the masses does seem pretty tactical and self-serving here.

Prerequisite for recursive self-improvement and far short of ASI, any conception of AGI really really needs to be expanded to include some kind of self-model. This is conspicuously missing from TFA. Related basic questions are: What's in the training set? What's the confidence on any given answer? How much of the network is actually required for answering any given question?

Partly this stuff is just hard and mechanistic interpretability as a field is still trying to get traction in many ways, but also, the whole thing is kind of fundamentally not aligned with corporate / commercial interests. Still, anything that you might want to call intelligent has a working self-model with some access to information about internal status. Things that are mentioned in TFA (like working memory) might be involved and necessary, but don't really seem sufficient

In the end though, it's probably about as good as any single kind of test could be, hence TFA looking to combine hundreds across several dozen categories. Language was a decent idea if you're looking for that exemplar of the "AGI-Complete" class for computational complexity, vision was at one point another guess. More than anything else I think we've figured out in recent years that it's going to be hard to find a problem-criteria that's clean and simple, much less a solution that is

Experimental design comes in here and the one TT paper mentioned in this thread has instructions for people like "persuade the interrogator [you] are human". Answering that a green eggplant is green feels like humans trying to answer questions correctly and quickly, being wary of a trap. We don't know participants background knowledge but anyone that's used ChatGPT would know that ignoring the question and maybe telling an eggplant-related anecdote was a better strategy

If you understand TT to be about tricking the unwary, in what's supposed to be a trusting and non-adversarial context, and without any open-ended interaction, then it's correct to point out homework-cheating as an example. But in that case TT was solved shortly after the invention of spam. No LLMs needed, just markov models are fine.

Ask your favorite SOTA model to assume something absurd and then draw the next logical conclusions based on that. "Green is yellow and yellow is green. What color is a banana?" They may get the first question(s) right, but will trip up within a few exchanges. Might be a new question, but often they are very happy to just completely contradict their own previous answers.

You could argue that this is hitting alignment and guard-rails against misinformation.. but whatever the cause, it's a clear sign it's a machine and look, no em-dashes. Ironically it's also a failure of the turing test that arises from a failure in reasoning at a really basic level, which I would not have expected. Makes you wonder about the secret sauce for winning IMO competitions. Anyway, unlike other linguistic puzzles that attempt to baffle with ambiguous reference or similar, simple counterfactuals with something like colors are particular interesting because they would NOT trip up most ESL students or 3-5 year olds.

I do like to think about the distinctions and boundaries for hard/soft/squishy knowledge though, and try to challenge assumptions and misconceptions about it. Invariably people have weird ideas about how "hard" their pet area is and how "soft" that thing they love to hate really is, which is itself a kind of dogma or superstition. Plus I think it's a public service to try and interest gear-headed nerds in things like criticism and philosophy (or vice versa, pushing engineering and math at the literature nerds). Last time I waded into this kind of debate I was pointing out that Frege worked on semiotics.

Crucially, entertaining ideas isn't the same as believing them, it's about giving them some time and space so you can work out whether it's consistent, rich, useful. Even in math this stuff is hard to get right, just look at the resistance and ridicule that Cantor had to go through, or look at the development of non-Euclidean geometry. And that's a space where proof is actually possible. Critical theory is a real thing but is always walking this fine line between being nonsense or being revolutionary.

> Being a gatekeeper by maintaining high intellectual standards is not what public opinion would associate with Social Text, to say the least. Yet that is what the journal practiced, mainly. And it is a practice worth defending, however elitist it might look. All the more so because of how the Trump administration has weaponized both the idea of the hoax and the program of anti-elitism. [..] We know what has befallen intellectual standards. [..] Is this ChatGPT, or is it Orwell’s doublethink?

Well ok, there's a conversation to be had about these things! This is not the time to pontificate though, it's the time for sweet revenge. There's never been a better time for po-mo wonks to lean on AI slop and blast physics journals with fake stuff about gravity until someone understaffed falls for the trick. Then you can do a big scandalous reveal about how you can't believe you got away it ;)

> If I would take S.P., I would hold Russia by the head. If I take Kiev, I will hold Russia by legs. If I take Moscow, I will reach right into its heart!"

https://history.stackexchange.com/questions/27588/why-did-na...

For reasons I don't claim to really understand, I don't think it even makes them deterministic. Floating point something something? I'm not sure temperature even has a static technical definition or implementation everywhere at this point. I've been ignoring temperature and using nucleus sampling anywhere that's exposed and it seems to work better.

Random but typical example.. pydantic-ai has a caveat that doesn't reference any particular model: "Note that even with temperature of 0.0, the results will not be fully deterministic". And of course this is just the very bottom layer of model-config and in a system of diverse agents using different frameworks and models, it's even worse.

I tried to raise this question yesterday. https://news.ycombinator.com/item?id=45683113#45687769

Declaring victory on "reasoning" based on cherry-picking a correct result about arithmetic is, of course, very narrow and absurdly optimistic. Even if it correctly works for all NxM calculations. Moving on from arithmetic to any kind of problem that fundamentally reduces to model-checking behind the scenes.. we would be talking about exploring a state-space with potentially many thousands of state-transitions for simple stuff. If each one even has a small chance of crapping out due to hallucination, the chance of encountering errors at the macro-scale is going to be practically guaranteed.

Everyone will say, "but you want tool-use or code-gen for this anyway". Sure! But carry-digits or similar is just one version of "correct matters" and putting some non-local kinds of demands on attention, plus it's easier to check than code. So tool-use or code-gen is just pushing the same problem somewhere else to hide it.. there's still a lot of steps involved, and each one really has to be correct if the macro-layer is going to be correct and the whole thing is going to be hands-off / actually automated. Maybe that's why local-models can still barely handle nontrivial tool-calling.