Generalization is a fundamental concept in machine learning that measures how well a trained model performs on new, unseen data. It is the goal of a machine learning algorithm to achieve high generalization, meaning it can accurately make predictions or decisions based on data it has not encountered during its training phase. The ability to generalize well indicates that the model has learned the underlying patterns in the training data without overfitting to noise or specific training examples. Effective generalization requires careful model selection, training procedure, and validation methods to ensure that the model is complex enough to capture the essential patterns in the data but simple enough to avoid fitting to irrelevant details.
Historical overview: The concept of generalization has been central to machine learning and statistical learning theory since their inception. The theoretical underpinnings of generalization were significantly developed during the 1970s and 1980s with the introduction of the Vapnik-Chervonenkis (VC) theory and the concept of VC dimension, which provide a framework to quantify a model's capacity to generalize.
Key contributors: Vladimir Vapnik and Alexey Chervonenkis were pivotal in the development of the theoretical foundations of generalization through their work on the VC dimension and statistical learning theory. Their contributions have laid the groundwork for understanding and improving the generalization capabilities of machine learning models.