Sampling in the context of machine learning and statistics is a fundamental technique used to reduce computational cost and simplify data management. It involves selecting a representative subset from a larger dataset (population) to make inferences or predictions about the dataset as a whole. This process is crucial in machine learning for training models on large datasets, where using the full dataset can be computationally expensive or impractical. Effective sampling strategies, such as random sampling, stratified sampling, and cluster sampling, ensure that the selected sample accurately reflects the distribution of the entire dataset, minimizing bias and variance in the model's predictions. Additionally, sampling is essential for techniques like bootstrapping, which relies on random sampling with replacement to estimate the distribution of a statistic, and Monte Carlo methods, which use sampling to approximate complex integrals in probabilistic models.
The concept of sampling has been used in statistics for centuries, but its formalization and widespread application in machine learning and data analysis became prominent in the late 20th century as datasets grew in size and complexity.
While it's challenging to attribute the development of sampling to specific individuals due to its foundational role in statistics, Sir Ronald A. Fisher and William Gosset (under the pseudonym "Student") are notable figures for their early 20th-century contributions to statistical methods that include principles of sampling.