Claude E. Shannon

Founded information theory and defined entropy as a measure of uncertainty.

Biography

An American mathematician and engineer whose 1948 paper created information theory in essentially complete form, giving precise limits on compression and on reliable communication over noisy channels. He also wrote one of the earliest serious treatments of computer chess.

Key contributions

• A Mathematical Theory of Communication (1948), defining entropy and the limits of coding.
• The source coding theorem, establishing entropy as the floor on lossless compression.
• An early formulation of computer chess based on game-tree search with heuristic evaluation.

Impact

Entropy is the unit in which uncertainty is measured across machine learning: information gain in decision trees and cross-entropy loss both descend directly from this work.