10 min readProbability Foundations
Entropy and Information
Measuring uncertainty in bits: Shannon entropy and why the logarithm is base 2, information gain worked on a split, and how cross-entropy and KL divergence relate to entropy and to the loss functions used to train classifiers.
Information TheoryProbabilityMathematics