Machine learning/Classification algorithms

Classification is a subcategory of supervised learning problems.

k-nearest neighbor edit

  • a simple classification algorithm
  • Intuition: Find the majority vote in the training data
  • This is a discriminative model, meaning that there is no way to generate the training data points

Algorithm edit

  • Define some distance metric or similarity metric. The simplest case is Euclidean distance.
  • Given some input point  , find the  'th nearest neighbors from the training set.
  • Do a majority vote between these nearest neighbor list and classify the input point as the category with highest number of vote.

Probabilistic interpretation edit

Consider the classification output as a random variable  . Define probability of   given input   and training data   is

The output of the classification is

Read more about probabilistic interpretation here: