# Week 7 notes: Support Vector Machines

2017-08-30**Introduction**

No notes.

**Large Margin Classification**

**Optimization Objective**

**Support Vector Machine**

Learn complex non-linear function.

*Source: Coursera Machine Learning course*

**Large Margin Intuition**

**SVM Decision Boundary**

Consider a case where we set constant C to be a very large value, when minimizing the optimization objective, we are going to be highly motivated to choose a value, so that the first term is equal to 0. So what would it take to make this first term equal to 0.

*Source: Coursera Machine Learning course*

When the first term is equal to 0, we need to minimize (ignored θ_{0}).

**Linear separable case**

The obtained decision boundary when minimizing the optimization objective will have the margin as large as possible (hence the name **Large Margin Intuition**).

This means SVM will choose the black decision boundary instead of the pink and green one:

*Source: Coursera Machine Learning course*

**Mathematics Behind Large Margin Intuition**

**Vector Inner Product**

`p`

= length of projection of `v`

onto `u`

. p can be positive or negative.

*Source: Coursera Machine Learning course*

**SVM Decision Boundary**

We can rewrite the optimization objective of SVM as follow:

s.t.

where p^{(i)} is the projection of x^{(u)} onto the vector θ.

Simplification: θ_{0} = 0.

According to the illustration below, with the minimal value of the magnitude of θ, the absolute value of p will large as much as possible (hence the large margin).

*Source: Coursera Machine Learning course*

More intuitive illustration:

*Source: Coursera Machine Learning course*

**Kernels**

It’s a technique.