Machine learning/Bird's Eye View

Here we give an overview and classification of machine learning topics and problems

Supervised Learning

Given training data $(x_{1},y_{1}),\cdots ,(x_{n},y_{n})$ choose a function such that $f(x)=y$

There are different categories of supervised learning problem

• Classification: $y_{i}\in \{{\text{finite set}}\}$
• Regression: $y_{i}\in \mathbb {R} ^{d}$

Unsupervised Learning

Given data points $(x_{1},\cdots ,x_{n})$ find patterns in the data.

Here, finding pattern is a general term which could mean

• Clustering
• Density estimation
• Dimensionality reduction.

Variations of supervised and unsupervised learning

Semi-supervised learning uses unlabeled data to refine decision boundary line

Semi-supervised learning

• Semi-supervised learning is an approach to machine learning that combines a small amount of labeled data with a large amount of unlabeled data during training.
• Unlabeled data, when used in conjunction with a small amount of labeled data, can produce considerable improvement in learning accuracy.
• This is particularly useful when there are a lot of unlabeled data and a small amount of labeled data
• The following figure shows how adding unlabeled data can refine the classification boundary

Active learning

• the learning algorithm can interactively query a user (or some other information source) to label new data points with the desired outputs.
• Active learning is similar to semi-supervised learning in the sense that there are a lot of unlabeled data (cheap to acquire) and a few labeled data (expensive to acquire).
• Active learning is sometimes called the optimal design of experiment

Decision theory

• General idea: "minimize expected loss"
• We take an action ($a$) based on the state of the system ($s$) and we define a loss function based on the outcome of the system and the world. The goal is to minimize the loss function which is defined based on the outcome of the system and the state of the world.
• Decision theory is closely related to the concepts used in reinforcement learning (RL) field (Markov Decision Process, etc.)
  • Maybe the difference between this topic and RL is that in RL we care about total lifetime rewards (planning), whereas here the rewards are more instantaneous.
• Different types of loss function
  • "0-1" loss: used in classification
  • "square loss": used in a regression problem

Reinforcement learning

• General idea: "Maximize lifetime rewards"
• Reinforcement learning (RL) is the process of optimizing rewards in a sequential decision-making process under uncertainty.

Generative versus Discriminative models

These models are used for classification.

• A discriminative model is a model of the conditional probability, $P(Y|X)$. Here $Y$ is the classification outcome and $X$ is the input data to the model.
  • The reason that these models are called discriminative is that it is only able to discriminate between classification outcome. In other words, once the model is formed, we cannot reconstruct the data behind the model. We are only able to discriminate the data and say what is the classification output.
  • A discriminative model ignores the question of whether a given instance is likely, and it just tells you the probability that an instance of the input data ($X$) falls into a category ($Y$).
• A generative model is a statistical model of the joint probability of input data,$X$, and label, $Y$: $$P(X,Y)=P(Y|X)\times P(X)$$
  • A generative model includes the distribution of the input data itself, and in this sense it is able to generate the data.
  • Generative model is more powerful and contains more details in comparison to Discriminative model, However, estimating the density of input data, $P(X)$ requires a lot of data and could be a source of error if we don't have sufficient data.