Tagged “vc-dimension”

2 articles.

14 September 20264 min readStatistical Learning Theory

One Parameter, Infinite Capacity

A classifier with exactly one real parameter fits all 1,048,576 labellings of twenty points, every time, and predicts a twenty-first at 0.5038 accuracy over twenty thousand trials. Counting parameters measures neither an upper nor a lower bound on what a model class can fit, which is why capacity has to be measured some other way.

Machine LearningMathematics

11 September 20267 min readStatistical Learning Theory

Why Learning From Data Works At All

The gap between the error you measure and the error you will suffer, why picking the best of a thousand identical hypotheses makes it look 0.1149 better than chance, how capacity is counted for infinite model classes, and the theorem that equalises every learner - with the assumption that makes it true.

Machine LearningMathematics