Self-Supervised Learning

Self-supervised learning trains a model on unlabelled data by turning the data itself into the label. Hide a token and ask the model to predict it; mask a patch of an image and ask the model to fill it in; take two crops of the same photo and require their representations to match. No annotator is involved, yet the training signal is as precise as a supervised one, which is what makes it possible to train on corpora far larger than anyone could ever label by hand. It sits between the two older categories rather than replacing them. Supervised learning needs a human-provided target for every example and is limited by how much labelling you can afford. Unsupervised learning looks for structure with no target at all. Self-supervision manufactures targets from the input, so it gets supervised-style gradients at unsupervised-style data volumes — the trick behind modern language models, most embedding models, and much of current speech and vision work. For builders the practical consequence is where the labels still matter. Self-supervision produces a general representation; turning that into a classifier, a ranker, or a well-behaved assistant still takes labelled examples, just far fewer of them than training from scratch would have required. That is why a small, clean, task-specific dataset is often worth more than a large scrape when you are adapting a model rather than building one.