Non-contrastive learning strategies, unlike their contrastive counterparts, do not rely on comparing similar (positive) and dissimilar (negative) pairs of data points to learn representations. Instead, these methods often focus on modeling the data distribution directly or optimizing representations based on properties inherent to the data itself, such as consistency under transformations or self-supervised prediction tasks. This approach can lead to efficient learning in scenarios where defining or sampling effective negative pairs is challenging or when the goal is to capture the broad structure of data without relying on binary distinctions.
Historical overview: The concept of non-contrastive learning has been around for several years, but it has gained more attention in the machine learning community in recent years, particularly with the rise of self-supervised learning techniques that do not require negative sampling, which can be seen in the late 2010s and early 2020s.
Key contributors: While non-contrastive learning is a collective development of the machine learning community, and no single figure or group can be credited with its inception, recent advancements in self-supervised learning techniques have been driven by researchers across several organizations, including academic institutions and tech companies. The names of specific contributors vary widely across specific methods and implementations within this broad approach.