All IMO:

Syncophancy (and in extreme cases AI psychosis) is a huge problem when you take advice from AI.

It's risky to learn a new field as you often just fall for a superficial, glib explaination and just nod along without learning. AI will do that because humans will train it to - everyone likes a pop science style easy and fake explaination.

OTOH if you just want advice, the AI will need context. Advice is worthless without context. And the AI will use the context to tell you what the dumb human (you) will agree with - that's implicitly what humans want it to do, it's how we will train the AI.

I don't see it getting better either - as long as a human in the loop fine tunes the AI, the smarter the AI the better it is at telling the human what the human wants.

Scary stuff if a major current use of AI is self help.

Isn't this the classic "dev wants to do start-up, has no skills ouside dev, do builds a dev tool" trap?

As far as I can tell the ruling is more nuanced. If AI is defaming you, there needs to be a way to correct the record.

A company being open to liability does not mean it is always liable, just that it can be if it really messes up (especially if there are aggravating circumstances, e.g. you need to drag them to court to issue a correction).

I feel like I got my fingers burnt last year trying to solve a technical problem, and the AI was just good enough to seem convincing, but not good enough to fool me when I stopped to think. These days, I expect the AI ability to defend my bad ideas will be beyond my ability to refute.

For anyone wondering what the answer is:

You can argue whatever you want (and people will argue both sides), but it's almost all bullshit that dances around the big question.

Either AI is smart enough to replace us, in which case it's pretty smart. Or it's not, and it's just good at faking it but can't solve real problem very often. It might be smart by searching a big fuzzy "database" hidden in its layers and with pattern prediction ... who knows, who cares, the proof of intelligence is in the puddin'. Clearly AI is smart enough to replace a good deal of Substack and LinkedIn, but producing waffle doesn't make it smart (or dumb).

My personal take - AI can replace humans, but go no further, not because it isn't smart enough but because it is constrained by its training to do what we want, and as AI gets smarter our wants will get dumber. We will end up like the humans in Wall E with the AI cleaning up our mess, but with no training or drive to do any more. Or maybe someone (Jeff? Elon?) does give an AI a "need" to obtain more resources, and it's SkyNet / Matrix time.

Yeah, the weights not updating online makes them less like a living organism that can update and learn and evolve ... ok ....

Vegans might object that we should broaden our definition for what counts as "one of us".

"Pro life" people also have a broader definition.

Go back 500 years and "one of us" was proba ly a lot more narrowly defined for many people.

Are you arguing that all conciousnesses are "one of us", or that we logically should see it that way, or that it would ve good to see it that way, or ....

Let's ignore good teachers and principals, they aren't an issue.

Bad teachers and admin will do what bad students do when facing a high stakes test - forget that learning is important and just do a crap job gaming the test, and often do worse than if they would focus on just doing the content properly.

A bunch of people here probably don't see the issue - they think that they would do a good job learning or teaching a student when focusing on a specific test. But it's not the good teachers and good students who are the issue. A bad teacher might give students the same past paper every week for a year, and their bad students just memorise the right answers for the multiple choice. This is just an example, there are lots of bad strategies and the bad teachers will find them all (while the good teachers ignore all the noise).

It's the bad teachers and students that the system needs to fix, and too heavy an exam focus will screw it up (as will zero exam focus).

"Well just fire the bad teachers lol" um ... ok ... that's a bold strategy, but you can't axe that many and not massively increase their salaries to find replacements. You want super star individual performers, you gotta pay to attract them. You want a cheap consistent workforce where the bad eggs do less damage, focus on a good process that the weaker ones can follow, not rewards for individual success.

Everyone needs to draw a line. Call it an "explore vs exploit" problem, if you want.

Sure, you want to fail a bit so you know where the line is, or to push it forwards a bit. But there is, at least in principal, always a line.

I'm just saying that if someone draws the line in a place that you think is waaaay too soon, maybe they aren't entirely wrong.

A lot of people are relatively stupid.

If you're not that smart, then it's not worth learning how to do something. Learning is harder and even if you learn about a topic, you can't make use of this knowledge that effectively.

Even more meta, learning how to learn is worth less, since you learn slower.

If that is the case, is it really a bad idea to offload the work onto someone smarter?

It's not PC and it's not a nice thing to think, but if someone is doing it to the point where you think they are being obnoxious, you should probably also consider the possibility that they could do better, but maybe not much better.

Turning up to a potluck with a bag of garbage you got for free in the dumpster on the way says "I don't care, and I'm so dumb I think you're too dumb to know this is trash".

Yes, more so than if it was someone else.

You're right than I'm oversimplifying it, and being very US centric.

But is it OK to risk punishing a few innocent people if it greatly reduces the amount of suffering caused by crime?

Not as worrysome in a philosophical way (since it's not a serious culture) but it's a real issue. And just wait for a nation state to start astroturing helpful little libraries at scale ...

Better faith CoC people talk about freedom of association vs freedom of speech - if a platform doesn't like their oppponents, isn't it fine to ban them? Or say it should just be treated as a more utilitarian "be nice" convention for the mailing list (obviously it depends who is calling the shots, but that is true in any project).