EGA: https://github.com/jcreinhold/ega (https://jcreinhold.github.io/ega/)

SGA: https://github.com/jcreinhold/sga (https://jcreinhold.github.io/sga/)

The list of apps that people most "want to delete" is highly correlated with apps with the list of apps with the most MAUs.

The right thing to look at here is the percentage of users who want to delete the app. Otherwise this table is just telling us that Instagram has more users than other apps—e.g., 1% of 1 billion > 1% of 1 million—which I suspect most people already know.

Fig. A in the original post corresponds to the same view (axial) of the brain as shown in the first image in [1].

The dark cashew-shaped blobs in the center of the healthy image in [1] correspond to what's labeled LV (for "lateral ventricles") in the original post. These cavities are filled with a fluid called cerebrospinal fluid (CSF) which serves to protect and clear waste from the brain.

Other examples of enlarged ventricles can be found, e.g., with a disease called normal pressure hydrochephalus (NPH) (example here: [2]).

So, needless to say, this case of ventricular enlargement is extremely severe, even compared to NPH.

[1] https://radiopaedia.org/cases/normal-ct-brain

[2] https://i1.wp.com/myneurosurg.com/wp-content/uploads/2017/11...

I agree that most of the value in a clinical application won't come from the often (but not always) relatively small performance gains by tweaking your neural network architecture or fiddling with the loss function. Collecting a high-quality and diverse dataset is important for training and arguably even more important for validation because you want to show that the deployed model is reliable.

But before deploying a model, I'd argue that it is worth testing a few architectures out to determine if one is substantially better than the rest. It can be a pain to test out a bunch of architectures, but the ones we mention in the article have many implementations freely available (and we provide ones too!). So you can drop in one of these architectures and test them out pretty easily (especially if you skip hyperparameter tuning initially).

Spending too much time fussing over a 2-3% performance gain is silly, but sometimes, surprisingly, the difference in performance by choosing another architecture can be much greater. I wish I had more intuition as to why some architectures perform well and others don't. It would certainly make R&D easier if you could totally ignore the architecture choice.

I agree that the mean and variance of the small batches won't be representative of the true mean and variance, but in practice, I've used batch norm for small batch sizes successfully (e.g., <8). In medical imaging, due to memory constraints, I commonly see batch sizes of 2 (or even 1, although it's not really "batch norm" at that point).

The paper "Revisiting small batch training for deep neural networks" [1] discusses the benefits of small batch sizes even in the presence of batch norm (see Fig. 13, 14). They only look at some standard CV datasets, so it isn't conclusive by any means, but the experimental results jive with my experience and what appears to be other researchers experience.

[1] https://arxiv.org/pdf/1804.07612.pdf

There is a set of more recent lecture notes here, https://web.ma.utexas.edu/users/gordanz/lecture_notes_page.h..., under the "Introduction to Stochastic Processes" section, FYI.