The Machine Learning Forum

Active Learning

The term active learning applies to a wide range of situations in which a learner is able to exert some control over its source of data. For instance, when fitting a regression function, the learner may itself supply a set of data points at which to measure response values, in the hope of reducing the variance of its estimate. Such problems have been studied for many decades under the rubric of experimental design [C72,F72]. More recently, there has been substantial interest within the machine learning community in the specific task of actively learning binary classifiers. This task presents several fundamental statistical and algorithmic challenges, and an understanding of its mathematical underpinnings is only gradually emerging.

Learning mixture models

Learning mixture models is a natural formalization of the clustering problem. A mixture model is a collection of distributions D₁, .., D_k and weights w₁, .., w_k. To sample from a mixture model, we draw i with probability w_i, and then draw a random sample from D_i. Thus, each cluster in the data corresponds to a different distribution D_i in the mixture, and the weights w_i correspond to the fraction of points from each cluster in the entire dataset. In the problem of learning mixture models, we are given samples from a mixture model, and our goal is to (a) learn the parameters of the distributions D_i and (b) classify each sample, according to its source distribution. The most popular algorithms for learning mixture models in practice are the Expectation-Maximization(EM) and the k-means algorithms.