In short: given some event of probability p, -log p = log 1/p is its "surprise". (If p = 1, log 1/1 = 0, so zero surprise; as p -> 0, the surprise gets bigger and bigger; and the surprise for two independent events, p = p1 * p2, is the sum of their individual surprises: log 1/(p1*p2) = log 1/p1 + log 1/p2.)

The entropy of a distribution is its average surprise: Sum/Integral of p log 1/p.

KL(p || q) is your excess surprise if you think something's distribution is q but it's actually p: Sum/Integral p (log 1/q - log 1/p). The KL divergence is always non-negative because surely if you think the distribution is q but it's actually p, on average you're going to be more surprised than someone who knows it's p.

Comments URL: https://news.ycombinator.com/item?id=13326938

Points: 1

# Comments: 0

Some things that have worked for me:

0. Goes without saying, but consistent effort. I've done at least a little French every day for the past two years.

1. Reading a lot. I started with the Harry Potters and now read for pleasure pretty much exclusively in French.

2. Reading on a Kindle. Instant dictionary lookup! This is such a big efficiency boost that I think that reading physical books is simply a mistake.

3. Listening a lot. I listen to about an hour of French podcasts/youtube channels a day.

4. Studying grammar. I mainly study grammar when I run into something tricky while reading, but I really do study.

5. Flashcards. I've only started making them in the past month or so, but yes, they really do work. I feel silly for not realizing that sooner. I highlight interesting words/expressions as I read and periodically dump them onto index cards.

For example, take a list of Strings. It's "more or less" a single string, it just hasn't made up its mind yet :) When you say fmap reverse myStrings, you're saying: yeah yeah, I know my string hasn't made up its mind yet, but just reverse it. So you reverse all the possibilities. When you say

  (++) <$> myStrings <*> yourStrings

Promises: A Promise Int isn't an Int, but eh, it's more or less an Int. Just as you can add 1 to a regular Int, you can fmap (+1) promisedInt. Just as you can replicate 3 'a' to get "aaa", you can

  replicate <$> promisedInt <*> promisedChar

Parsers: A Parser Int (something that can gobble up a string and produce an Int) isn't an Int, but (in some contexts, if it's helpful) you can think of it as being more or less an Int. Saying fmap (+1) intParser says make a parser that parses whatever intParser does, and then adds 1 to the result. Doing

  replicate <$> intParser <*> charParser

Functions that take some fixed argument type, say Float. A Float -> String isn't a String, obviously, but it's more or less a String :) It's just missing a Float. If I say fmap reverse justNeedAFloatToBeAString, I'm making a new function will take a float and then reverse whatever you get by feeding it to justNeedAFloatToBeAString. If I

  (++) <$> almostAString <*> almostAnotherString,

Functor says you can treat a single f a as more or less a single regular a. Applicative says you can treat any number of f a values as more or less any number of regular a values. Monad says you can do control flow with f a values as if they were regular a values.

Comments URL: https://news.ycombinator.com/item?id=7805947

Points: 1

# Comments: 0

Comments URL: https://news.ycombinator.com/item?id=7802437

Points: 2

# Comments: 0

[0] https://github.com/happy4crazy/ittybit/blob/master/src/ittyb...

[0] http://classicprose.com/ [1] http://video.mit.edu/watch/communicating-science-and-technol...

(It seems like you've got interesting things to say, but you're being a bit strident. It bums me out that Scala discussions on HN take such a heated tone.)

How do you get your code reviewed? How do you review other people's code? Do you use specific tools/sites/mailing lists/IRC channels/friends?

Comments URL: https://news.ycombinator.com/item?id=5146214

Points: 8

# Comments: 6