You could even write an article asking why that original article was even written, and it might make for more interesting content.

I dare not even read the rest of the page just in case my brain accidentally absorbs other bad information like that paragraph about GANs.

In a stats textbook, when you know that your training data comes from a normal distribution, you can maximize the MLE wrt the parameters, and then use that for sampling. That's basic theory.

In practice, it was very hard to learn a good pdf for experimental data when you had a training set of images. GANs provided a way to bypass this.

Of course, people could have said "hey let's generate samples without maximizing a loglikelihood first", but they didn't know how to do it properly, how to train the network in any other way besides minimizing cross-entropy (which is equivalent to maximizing loglikelihood).

Then GANs actually provided a new loss function that could be trained. Total paradigm shift!

Conceptually, it's easier to think of "music from high school" than about the specific mix of subgenres from my playlist back then. Same for documents that I saved. Those ICQ logs from high school are there. They don't belong in the same folder as the stuff I wrote yesterday, even though they could be of a similar nature.

Email ends up being the form of online identity for a lot of people, myself included, so that almost every service that I sign for has my email address as ID. If that email address isn't the ID, it's the preferred way of resetting passwords. I wouldn't be super happy about Facebook being my online ID, nor my cell phone number (see SIM swapping problems).

It's life changing in the same way that losing all your personal documents in a fire sets you up accounting nightmares. Moreover, you're making very light a situation about losing all your pictures. I'm not talking about food pictures, but there's plenty of "me" that's contained in being able to look at pictures of important events of my life (which is why I don't rely only on cloud backups for that).

I don't know what's "life changing" to you, then.

There isn't much to say to respond to that, apart that it seems to me that, in a parallel universe, you might have had a more fulfilling experience, or you might have cut your losses and walked away sooner.

That's the cruel aspect of the PhD. It really seems like a lot of important things are outside of one's control, especially when it comes to important factors in mental health. Nobody's starting a PhD with the goal of sinking hours into Reddit and Buffy because they feel awful about their PhD experience.

If he had indeed tried to pull that trick in 2020, a lot of people would have remembered that he said he was setting it up.

He said that for a good number of years, every time before going to bed, he would write on a little card that he predicted he would die that night during his sleep. In the morning, he got up and happily threw away the little card. Every day. For many years. His concept was that, on the rare chance that this actually happened to be last day, people would think that he pulled the ultimate magic trick. People would not suspect that he wrote this on a little card every day because, well, nobody does that.

Given Google's reputation to ditch their own products, I guessed this was some side projects that some Googlers did, and it was never in Google's main strategy to allow people to sync their Google Drive to their local machines. Quite the opposite, actually.

My current gripe with Dropbox is that I'd like to basically be able to pay 4x the "Dropbox Plus" cost in order to get 4x the storage (without having to manage 4 separate accounts). Having 2TB isn't enough, but having "infinite" with Dropbox Business certainly is more than I want.

His point was mostly that, way before you achieve the kind of AGI portrayed in fiction, you'll have semi-intelligent interdependent systems that cause a lot of trouble due (like the kind that already happens to a lesser degree). Those are the ones that we should worry about right now.

One day he participated in this "split or share" kind of experiment, and he was ruthless. Nobody's emotions would be damaged by acting nasty and never sharing with the computer program.

Turns out, it probably was actually a real person who was behind on the other side. He saw some old woman crying, coming out of some adjacent room after the experiment was over.

So, yeah, different social conventions definitely apply.

That experience completely differs from mine. Maybe I just prefer email to text or Facebook messages, but the same principle applies to wherever you are writing something to friend or family. Wouldn't you get the same problem, just elsewhere?

Responding to my mother's birthday wishes does seem like it's a different kind of activity than autocompleting C# code, even though both can be executed with autocomplete to get a good valid output.

When you're building digital circuits, they're expected not to care about what the bits mean, which patterns are more likely. It works for all possible inputs, with equal quality.

There are things in common with how you would process faces and how you would recognize other visual objects, and that's why there are design patterns such as "convolutional layers come before fully-connected layers".

In a way, the "no free lunch" theorem says that you are always paying a price when you specialize to a certain kind of patterns. It comes at the detriment to other patterns. So, any kind of stack of theories on ML/DL is going to be incomplete unless you say something about the nature of your data/patterns.

(That doesn't mean that we can't anything useful about DL, but it just puts a certain damper on those efforts.)

If I issue some kind of "IOU" certificate, redeemable for a rare Charizard card, and people trade those IOUs instead of redeeming them (good thing because I don't have those Charizard cards in my possession at the moment), then I'm basically expanding the supply "things that people trade and commonly use to pay for goods" (i.e. money/currency).

I didn't create any new Charizard card, but as long as people don't ask me to redeem them, it's roughly as though the market had 10 more copies of that card circulating.

There isn't more "value/wealth" created, but there is now more "money/currency" circulating. You can imagine how something similar is happening with derivatives.

You can’t fault the Dropbox people there for making a reservation and expecting that it would be valid. They’re a bit clueless in how they respond, though, not realizing that those rules are clashing with the unofficial social dynamics happening there.

(Semi-related : That’s why we might feel that banks are assholes for foreclosing houses that belong to deployed soldiers. Legally they can do it, but it sounds like it’s the shittiest application of the law.)

But ... I have the feeling that the author, who is relatively new to the field (by his own admission), expanded a lot of formulas and made certain parts of the theory more complicated than it should be.

Look around page 60. There are formulas with 6 summation signs in front of them, with all kinds of little indices floating around. How about page 37 ?

In a way, the whole point about the chain rule (and software libraries that implement it) is that you can stay in "math world" to do the reasoning, and not think about the job of managing the computation.

Same idea with expression as much as possible in terms linear algebra primitives. Matrix multiplication is easier to understand when it's not broken apart into sums whose indices you have to track.

You train many models. Then you "distill" their predictions into one model by using the multiple predictions (from many models) as targets (for the single model trained afterwards).

You're right to point out that humans don't do that.

I think it would be "cheating" if you train BetaGo on AlphaGo, for the purposes for doing that experiment. The goal would be to have some kind of "clean room" where people fumble around.

Of course, you can also run the other experiment to see how fast you can bootstrap BetaGo from AlphaGo. That's also interesting.

The problem is that there are about 3-5 alternatives out there, and none of them are mature enough or convincing enough to dominate. The field changes so fast that it's easy for them to become obsolete.

What you're seeing here is the enthusiasm of people who really want to get a good tool with proper support, and be able to stick with it. I'm still not sure if TensorFlow is that tool, but it depends on what will happen to it during the coming years.