If that's not the case then it might just trigger bad risk compensation behavior in the model's human operators.

My best guess is that somewhere close to the root of the problem is that language models still don't really distinguish syntagmatic and paradigmatic relationships. The examples in this article are a little bit forced in that respect because the alternatives it shows in the illustrations are all paradigmatic alternatives but roughly equivalent from a syntax perspective.

This might relate to why, within a given GPT model generation, the earlier versions with more parameters tend to be more prone to hallucination than the newer, smaller, more distilled ones. At least for the old non-context-aware language models (the last time I really spent any serious time digging deep into language models), it was definitely the case that models with more parameters would tend to latch onto syntagmatic information so firmly that it could kind of "overwhelm" the fidelity of representation of semantics. Kind of like a special case of overfitting just for language models.

In 1994, though, an 11 year old computer would already be considered vintage. In 1983 the hot new computer was the Commodore 64. In 1994 everyone was upgrading their computers with CD-ROM drives so they could play Myst.

Worth noting, too, that the paper outlines several possible experiments. It also specifically mentions some relative shortcomings of the model, and lists existing observations that they haven't tried to reconcile with it yet.

In fact, it almost seems like the only people able to produce great things in the 1970s were massive entrenched corporations like Ma Bell.

Funny, that.

Come to think of it, wasn't there a much more vibrant browser ecosystem in the late 90s and early 2000s, before Google used its dominant position in the ad market to undercut the competition? There used to be a lot more mobile operating systems out there, too.

I wonder what happened to all that competition? It's almost like some sort of massive anti-competitive influence came into force in the tech scene somewhere in the 2000s. . .

In general I'm a big fan of the "single wringable neck" principle. I've seen it put to great effect in the hands of a skilled leader, in both Scrum and non-Scrum teams. Better yet, when the leader isn't managing things well, it also leaves no question that they're the one who needs to figure out how to set things straight. Same goes for their delegates.

And for the ICs it makes collaboration easier - and therefore, ironically, enables them to work more autonomously. When everyone unambiguously knows what they're in charge of and what the team's big-picture objectives are, they have everything they need to independently figure out how best to make it happen. And it's also a lot easier to figure out who to talk to when they need to call attention to a problem.

I've seen a lot less luck with shared authority and informal delegation. On the best of days, it turns decisionmaking into an unnecessarily political process. More likely, the team will settle into an informal consensus process that typically operates as "rule by the obdurate" in practice. And when things get tough, the leaders will tend to slide into unproductive bickering that all but precludes actually fixing the problem.

Favorite readings that touch on this kind of thing: The Tyranny of Structurelessness by Jo Freeman, and Turn the Ship Around! by David Marquet.

But here are some examples of things that used to fall under the AI umbrella but don't really anymore:

  - Fulltext search with decent semantic hit ranking (Google)
  - Fulltext search with word sense disambiguation (Google)
  - Fulltext search with decent synonym hits (Google)
  - Machine translation
  - Text to speech
  - Speech to text
  - Automated biometric identification (Like for unlocking your phone)

Microsoft's been trying to ram essentially that down my throat for the better part of a year now, and it's mostly convinced me that the answer is "no". I don't want to have arbitrary conversations with my computer.

I still just want the same thing I've been wanting from my digital assistant for 30 years now: fewer "eat up Martha" moments, and handling more intents so that I can ask "When does the next east-bound bus come?" and it stops answering questions like "Will it rain today?" as if I had asked "Is it raining right now?". None of those are particularly appropriate problems for a GPT-style model.

Which tells me the problem may be fundamental, not a technical one. It's not just a matter of needing "more intelligence". I don't question the intelligence or skill of the people on the outsourced team I was working with. The problem was simple communication. They didn't really know or understand our business and its goals well enough to anticipate all sorts of little things, and the lack of constant social interaction of the type you typically get when everybody's a direct coworker meant we couldn't build that mind-meld over time, either. So we had to pick up the slack with massive over-specification.

The best defense against this that I've found is to ensure, as much as possible, that interfaces can be replaced. The single responsibility and interface segregation principles can help here. Using small, focused interfaces and letting modules implement more than one of them makes it easier to use the strangler pattern to replace interfaces that no longer work well with new and improved ones.

Also avoid temporal coupling as much as is feasible. Unnecessary statefulness is the easiest way to make this sort of thing harder than it needs to be.

The ones where classes were a breeze and barely challenging at all were the private college prep school and the not-quite-as-rich-but-still-pretty-wealthy high school in the suburban white flight community. My read on the situation was that teachers had figured out that every single parent believed their kid deserved a constant stream of gold stars, and had the leisure time and resources to make their lives miserable until it started happening. Also the white flight school cared a lot about its football team so you better make for damn sure that the classes aren't so challenging that the quarterback has trouble balancing their time spent on homework, practice, and partying.

Our city has a school choice program that includes a portal where you can look up these kinds of quantitative measures, and I think I agree with him. Tiger parents slosh from school to school as they chase after rankings, and, much like ill-contained liquid cargo in ships, all that motion tends to destabilize and capsize schools.

Sadly, I don't think smaller higher education institutions can afford to take such a relaxed attitude about it. They don't get to have an enrollment backstop in the form of a semi-captive audience of parents who live nearby and aren't hyperactive enough to commit to spending upwards of an hour every weekday trucking their kids back and forth across town.

Lack of typeclasses is a good example. Interface inheritance isn't my favorite, but after years working as the technical lead on a Scala project I've been forced to concede that haranguing people who just want to do their job and go home to their family about how to use them properly isn't a good use of anyone's time. Everyone comes out of school already knowing how to use interfaces and parametric polymorphism, and that is fine.

My understanding is that, for the purposes of determining fair use, a derivative work is substantially the same thing but in a different format. Transformative work must involve significant additional creative contribution "Changing the medium of a work is a derivative use rather than a transformative one." They cite previous case law that holds repackaging a print book as an e-book as a "paradigmatic example of a derivative work." The law also offers some paradigmatic examples of transformative work, such as criticism, commentary and scholarship.

Based on all of that, I would guess that, for the purposes of copyright law, a JPEG of a painting is absolutely a derivative work and not a transformative one.

And I've only got a little bit of time in this life; I'd much rather read a trashy romance novel that was written this year and meant to entertain me than the trashy romance with politics that make me cringe that my mom was reading 50 years ago.

This is why, for example, the Library of Congress doesn't just keep a copy of everything. It's not just a space constraints or storage costs issue; it's a signal-to-noise ratio issue. As Mark Crislip is fond of saying, when you mix apple pie and cow pie it doesn't make the cow pie better, it just makes the apple pie worse.

When you get down to it, this is just not a good sphere for deontological ethics.

It also discusses that there is a very good reason why digital libraries don't typically get to have perpetual rights to a work at the retail (or used) price for a print book. Basically, physical books wear out with use, ebooks don't, so there's a built-in mechanism for revenue recurrence that happens with print books but not ebooks. The ruling points out that publishers originally sold ebooks to libraries at the same pricing as print books, but abandoned the practice because they discovered that it was not financially sustainable.

And that's ultimately where the harm comes in. The IA is trying to create a loophole that subverts the income stream of all the people who work on a book by offering derivative works - which are never fair use; fair use is for transformative works - without paying the market's customary price for acquiring rights to create and distribute derivative works.

(As an aside, when I see authors speaking for themselves on these sorts of issues they will typically point out that editors and typesetters and cover artists and all the other folks who work on a book also deserve to get paid. It seems to only be people who are tokenizing authors for rhetorical purposes who want fixate on authors specifically and erase the value-adding contributions of "the publishers".)