I don't even know when I watched the shitty lightbulb head Sora clip but that feels so long ago now and nothing?

I just want to make crazy experimental AI film no one will watch. What is the hold up?

Just waiting for "This technology is just too dangerous to release before the US elections" --Sam Altman

These processes aren't cyclical, they are non-ergodic, one off regimes that once we visit a specific regime, we will not return to that regime.

As someone who has done quite a bit of t-bill and chill investing this regime, I don't see how we can afford to lower interest rates anywhere near like we did the last decade. I am good with 50 bps but it can only go so low this time to keep US government debt attractive.

Then I think of how I could have been a really shitty front end React developer circa 2019. There is no way I could ever get a job doing that now when I see what I can do with Cursor/Sonnet as the same shitty React dev.

I suspect it would be just as easy to write a paper that saying please has absolutely no effect on the output. I feel like gpt4 is/was stochastically better on some days and at some hours than others. That might even be wrong though too. The idea that it is provable that "please" has a positive effect on the output is most likely a ridiculous idea.

I would have had no problem intellectually getting through the program but quit after the first night in a hospital.

Anyone sitting at a desk can not understand how tough and miserable a nursing job is. Everyone is basically miserable and stressed out. The work is completely thankless, disgusting and dangerous with personal liability on the line if you make a mistake. Everything that we take for granted in an office setting just doesn't apply in a medical setting.

I eventually just went back to a bullshit project management job, for more money than a nurse of course. This is obviously part of the problem.

It is easy to complain about the system when it is someone else who has to help grandma to the bathroom. There is no easy solution for any of this given the demographics. It is basically a disaster.

I am a non-software engineer, I work for a small company that we could do whatever we want with language models in terms of the business. The issue though is all of our business problems are stacks of one off problems that the current models offer us basically no help on. Even if the models could help us we would basically have to rebuild the business from the ground up around the models to really benefit much.

Like the dotcom bubble, there is a large amount of time it will take for culture , business and society to leverage the technology. Between then and now I suspect sits an absolutely brutal AI winter that is brought on by nothing more than a good old recession.

Statistical Rethinking by McElreath is really what finally got me to see the value of R.

You can find the python versions of the class and they are certainly not better.

A full application in R really makes absolutely no sense.

We already have this field. It is just too new though. We seem to conflate the sciences with the efficient market hypothesis instead of the reality of progressing one funeral at a time. We may even have structures now that keep things in place after the funeral and stall progress even further.

I study complex systems on my own but if I was a PhD student I am not sure it would be a great career move.

I am not sure we could do that today. I think the push would largely to be to ban the automobile completely as too dangerous if we had to do it in 2024.

This is all just a mirror being held up to such a wealthy society and the changing risk preferences that come with that over time.

The distance runner or cyclist is not obese in your example.

There is simply no way to exercise enough if obese to create the caloric deficit needed.

This is all really part of the problem. Telling obese people to exercise more and don't restrict calories too much just doesn't work. What works if obese is a massive calorie deficit over time because an obese person has a massive amount of calories stored by definition.

It is just a much different situation compared to a distance runner that is already very learn. That person does have to worry about not consuming enough calories exactly because they don't have the stored calories the obese person does.

Of course, it is an overly broad stroke that doesn't quite capture all the nuance of the model but the alternative of "come on guys, just admit the model is thinking" is much worse and has much less to do with reality.

It might be like trying to train a neural net in 1993 on a 60mhz Pentium. It is the right idea but fundamental parts of the system are so lacking.

On the other hand, I worry we have gone down the support vector machine path again. A huge amount of brain power spent on a somewhat dead end that just fits the current hardware better than what we will actually use in the long run.

The big difference though from SVM is this has captured the popular imagination and if the tide goes out, the AI winter will the most brutal winter by an order of magnitude.

AGI or bust.

From a user perspective too, I was a subscriber from the first day of gpt4 until about a month ago. I thought about subscribing for the month to check this out but I am tired of the OpenAI experience.

Where is Sora? Where is the version of chatgpt that responds in real time to your voice? Remember the gpt4 demo that you would draw a website on a napkin?

How about Q* lol. Strawberry/Q*/o1, "it is super dangerous, be very careful!"

Quietly, Anthropic has just kicked their ass without all the hype and I am about to go work in sonnet instead even bothering to check o1 out.

We shouldn't expect the stochastic parrot to be able to do this though and it is unfair to the stochastic parrot.

It is like expecting a real parrot to say words it has never heard before.

No one asks that of a real parrot because we don't anthropomorphize a real parrot like we do the LLM.

The magic is in the fact that the model can't say "I don't know". The model would have to have arbitrary thresholds and a type of domain/context classification in order to set this arbitrary threshold for the conversation in order to say "I don't know". It would create all these unsolvable boundary conditions. AGI in this context then would be minimizing the need for "I don't know" until it no longer applies. Is that possible? I don't know :)

I would defer to Chomsky also on the subject that we are basically nothing like these models when it comes to language and we are not auto-completers.

I think we really need a new word for this process because it really is just not comparable to anything previously.

Unfortunately, "hallucinate" is a horse that has left the barn with seemingly no possible way of replacing the horse at this point.

If Page, Carmack and Zuckerberg were all trying to do something in nuclear physics I think they would have been where young people are today with computing.

A past problem with the low hanging fruit picked and continued progress having undesirable higher order effects.

I am sure their are just as brilliant young people today. If I had to bet, I would bet the reason we don't feel this is because they are in a domain that is not purely computer science related and a really good chance what they are doing is not in English but Mandarin.

What I highly overestimated was people's thirst for knowledge. I really thought that by this time I would be unemployable as the average young person would just be so learned that I wouldn't be able to keep up. The advantage of growing up with the internet would be just so huge.

I would have never guessed that the average young person instead almost has a type of learning disability from being addicted to political nonsense, stupid videos and gossip.

Everything is there from what I envisioned in 1995 though. It is just this useless, pernicious aspect dwarfs what I envisioned then in terms of popularity.

Recording time use to cost a huge amount of money and then DAWs put that in the hands of basically everyone with a PC.

We got an infinite amount more of half finished demos that no one listened to. I would have a hard time saying with a straight face that music as a whole has got better. The sheer volume crowds out a lot the fringe from being worth the effort of creating.

AI Art is really a better example though. I just resubscribed to midjourney this weekend on the web. I could see the thousands of images I made on discord. I think some are cool but what use are they? Millions of synthetic photo realistic selfies that no one but the creator bothers to look at.

The arts ultimately need a network of people appreciating the art form or you have nothing. Just an infinite amount of board classroom notebook doodles that no one ever sees. It doesn't matter if it is Picasso doing bored doodling if there is no audience. You can't have all artists with no audience. Really good chance with no audience Picasso just does something else too.

I mean has the art world really been disrupted by midjourney? It is an absurd idea. What is most curious is how little disruption any of this seems to be having.

Beyond that, the previous chats could practically be deleted for me. Its not like my prompts are such works of art that I couldn't recreate the thought if needed. Not to mention, Sonnet has made me not use chatGPT for about a month now so the blog is nicely divorced from the LLM flavor.