The trouble is we've already had a web flooded with "ai content" long before GPT was public. Plenty of young writers have been trained to churn out thoughtless streams of writing based on prompts that appear to be written by an intelligent mind but are often filled with meaningless non-sense.

My industry specific example is Towards Data Science, content created by fleshy AIs that often looks very insightful at first glance, but when viewed by an expert ends up being mostly incorrect gibberish.

H2O + (Solar) Energy + CO2 => Useful hydrocarbons (everything form sugar to gasoline)

O2 + Hydrocarbons => Useful Energy + CO2

When you burn a log you're really using a solar battery that took potentially decades to charge and roughly 25x the energy you feel from the fire (photosynthesis is only 4% efficient).

The same process that feeds us powers the global economy, but at the cost of emitting CO2 in direct proportion to the energy we're using and benefiting from.

This is why "miracle" solutions are so unlikely. Because they require a major disruption of this process in a way that it's not clear is fundamentally possible. Anyway to massively remove CO2 from the atmosphere is fundamentally going to require energy. And because of the nature of inefficiency, will always be a poor use of any energy you used to create the problem.

This is obviously where things like nuclear fusion do provide the possibility of breaking this process because they create a lot of energy outside of solar powered carbon cycle.

In fact because you need people to get set up remotely, I find the documentation tends to be better at all remote companies. In-Office companies sort of assume that you can just tap someone on the shoulder if you get stuck so there's more often, in my experience, gaps in the documentation.

I particularly find this a strange claim since open source projects have been successfully onboarding new people remotely prior to there even being efficient ways to screen share/video chat etc.

If you can run a company on the go or at home, certainly I am capable of shipping quality code at home.

My experience is that all of the really intelligent people interested in crypto did leave after the ~2012 wave of excitement. At that time when you saw people give talks on crypto they were almost entirely technical with very little focus (or interest) on becoming rich. That was when the people involved tended to be technical idealists. I didn't buy that crypto was the future then, but I wanted to be wrong.

Fast forward to ~2017 during the next crypto boom and the conversation was around the non-technical people at technical companies getting excited. People did believe crypto was going to become the currency of the world, so there was still some idealism, but it was mainly about getting in to get rich. By this wave a good chunk of the idealists I knew were entirely disillusioned.

Then the 3rd wave which just happened to correlated with a massive injection of money in the market by the Fed. At this point it was just literally get-rich-quick dreamers with more dollars in their hands than sense. Nobody I know who has gotten in during this period even has a coherent vision of what the future looks like, they just have too much money and think crypto is the way to get insanely rich one day. It's also when people completely unrelated to tech started getting involves. People who don't even know how to use a wallet, and rely 100% on 3rd parties to manage all of it.

People are holding on to crypto for the same silly reasons that coworkers of mine keep all their vested stock in companies that have dropped 50%+ in value over the last year. It's because they earnest believe that the era of low interest, free money is the norm. They believe this current macro is just a blip, and if they just hodl a bit longer it everything will go back to "normal".

As an example: if you want to implement logistic regression in JAX, you need to optimize the weights. This is easy enough since this can be modeled as a single value, a matrix of weights. If you want to model a 2 layer MLP, now you have to use 2 matrices of weights (at least). You could treat this as two parameters to your function (which makes the derivative more complicated to manage) or you could concatenate the weights and split them up, etc. Annoying, but managable.

When you get to something like a diffusion model you now need to manage parameters for a variety of different, quite complex, models. It really helps if you can keep track of all these parameters in whatever data structure you like, but also trivially just call "grad" with regard to these and get your models derivative with respect to its parameters.

Pytrees make this incredibly simple, and is a major quality of life improvement in automatic differentiation.

I highly recommend checking out Hamming's lectures if you find this enjoyable.

0. https://www.youtube.com/watch?v=uU_Q2a0S0zI

We also, so far, globally have not shown any evidence of replacing fossil fuels with green energy, only supplementing them.

a(b(c(d))) vs d |> c |> b |> a

but I'm not convinced pipes are better than more verbose code that explains each step:

step1 = c(d)

step2 = b(step1)

result = a(step2)

I've written a lot of tidy R and do understand the specific use cases where it really doesn't make sense to use the more verbose format, but generally find when I'm building complex mathematical models the verbose method is much easier to understand.

0. https://en.wikipedia.org/wiki/Open-source_intelligence

AI/ML products fundamentally don't make sense compared to products that happen to use some AI/ML to aid in solving a problem.

It's sort of like loving to use redis (which I do) and thinking you want to found a company based on using redis in the product, or start a redis product team, dedicated to shipping products that use redis.

It's one thing if you want to host redis as your business, which is solving a problem involving redis, but if your aim is to use redis to solve a problem then you're going to be in trouble.

Imagine a PM on the "use redis" team rejecting a great idea for customers because it could be more efficiently solved using a traditional database, or forcing the use of redis when a cheaper, easier solution already works just as well if not better. This is actually the case on AI/ML teams.

GPT startups that will thrive are the ones that aren't GPT startups, but instead solving some other, real, problem that happens to only be solvable in a post-GPT word.

Massive Neural Nets do require a lot of data and are often not the best solution, but differentiable programming in general does not have higher data requirements than manually computing your derivatives or using OLS. You can still approach classical ML from the perspective of differentiable programming (and likely end up with a better sense of our how your models work in the end).

While it would be great if everyone interested in the topic was well versed in the fundamentals, the truth is if you want to do anything from building something cool over the weekend to getting an actual job doing AI work, you're much better off starting not only with ML, but specifically with current SotA neural networks.

If you really want to get started in AI I highly recommend building even a trivial implementation of Stable Diffusion on your own. Not just because it's cool, but because at its heart it is an excellent demonstration of how current differentiable programming works. Diffusion models involve chaining together 3 separate models into an entire system that learns to solve a complex task. Once you understand this deeply, you can now solve a very broad range of tricky problems and are really approaching what we think of when we think of AI.

Differentiable programming is really the current pathway to any sort of AI solution to a problem.

I say this as the token "have you tried logistic regression?" guy in my org.

It seems like if the goal is to "play around with vector databases", why not just install it on your local machine? Part of using these tools is learning how they work and configuring them yourself.

If the goal is "start developing products using vector data bases" then it seems like you would surely want something a bit more under your control than using replit.

While I agree that this is a naive belief to have (at this point in my career I think there's almost a slightly negative correlation between skill and TC) the general talent pool for software engineers has, at least in my experience, dropped tremendously while TC has exploded.

The most important skills for getting high paying jobs in the last few years has been grinding leet code, then grinding systems design etc, etc. Software engineers no longer have "rare/valuable" skills, they have highly commodified, easily replicable skills (at least at the interview level).

Software engineers today simply aren't that skilled (at least on average) despite what they want to believe. It reminds me a lot of dotcom bubble where anyone with a pulse that could turn on a PC could get a high paying job.

As someone with a lot of time spent experimenting with programming languages, I can't think of any that can be so excellent from a research/experimentation perspective that also share the real production usage that Haskell sees. Take for example Racket, an amazing and also extremely flexible research language. It's probably easier to get started with than Haskell, but has never seen the real world usage that I've seen from Haskell.

Specifically with airfare, a 3rd party is not allowed to sell for less than the airline directly, so it's always better options since it is much easier to reschedule/cancel/refund directly with the airline. Plus, if you travel a lot, it is better to find a favorite airline and stick to them. Any bonus "features" offered by a 3rd party I can assure you are either not in your interest or actually a scam.

I don't know if the pricing rules applies to hotels, but I'd rather pay extra then get to the hotel and be screwed over last minute because some 3rd party is trying something "clever" behind the scenes and it turns out it ruins your travel plans.

I've worked at a pretty wide range of tech companies throughout my 15+ year career. When I was young I would ask myself "what is leadership thinking!?", my realization later in my career was simply: "oh, they're not thinking"

14 year old and younger tech companies have only existed during a tech boom period, when money flowed easy from both investors and customers. There has been zero market pressure to put thought into building products.

In the case of dropbox in particular, any long term customers (such as myself) can confirm that there has clearly not been a coherent product strategy for at least the last few years.

What's really wild is that the entire Apollo program cost ~$25 Billion in 1973 (of course est. $163 billion today) while Meta already spent $36 Billion on the Metaverse [0].

0. https://www.businessinsider.com/meta-lost-30-billion-on-meta...