# 支持向量机(SVM)之硬阈值

2021-06-24 00:44:11

### 一、超平面 ( hyperplane )

$b + w_{1}x_{11} + w_{2}x_{12}=0$

$b + w_{1}x_{11} + w_{2}x_{12} + w_{3}x_{13}=0$

$b + w_{1}x_{11} + w_{2}x_{12} +...+ w_{p}x_{1p} = 0$

$b + w_{1}x_{11} + w_{2}x_{12} +...+ w_{p}x_{1p} = 0$

$b + w_{1}x_{11} + w_{2}x_{12} +...+ w_{p}x_{1p} > 0$

$b + w_{1}x_{11} + w_{2}x_{12} +...+ w_{p}x_{1p} < 0$

### 二、使用超平面进行二分类

$b + w_{1}x_{i1} + w_{2}x_{i2} +...+ w_{p}x_{ip} > 0,\quad if \quad y_{i}=1 \\ b + w_{1}x_{i1} + w_{2}x_{i2} +...+ w_{p}x_{ip} < 0,\quad if \quad y_{i}=-1$

$(b + w_{1}x_{i1} + w_{2}x_{i2} +...+ w_{p}x_{ip})y_{i}>0$

### 三、最大间隔阈值（$$maximum\quad margin$$）表示

$distance = \dfrac{|Ax_{0}+By_{0}+Cz_{0}+D|}{\sqrt {A^{2}+B^{2}+C^{2}}}$

$distance = \dfrac{|w^{T}x_{i}+b|}{w^{2}} = \dfrac{|w^{T}x_{i}+b|}{||w||}$

$(b + w_{1}x_{i1} + w_{2}x_{i2} +...+ w_{p}x_{ip})y_{i}>\beta$

$\mathop{argmax} \limits_{w} \dfrac{2}{||w||}\quad \quad s.t.\quad y_{i}(w^{T}x_{i}+b)>1,\quad x = 1, 2,...,m$

$\mathop{argmin} \limits_{w} \dfrac{1}{2}||w||^{2}\quad s.t.\quad y_{i}(w^{T}x_{i}+b)>1,\quad x = 1,2,...,m$

### 四、对偶问题与KKT条件

$L(w,b,\lambda) = \dfrac{1}{2}||w||^{2}+\sum_{i=1}^{m}\lambda_{i}(1-y_{i}(w^{T}x_{i}+b))$

$\dfrac{\partial{L(w,b,\lambda)}}{\partial w} = w-\sum_{i=1}^{m}\lambda_{i}y_{i}x_{i}=0\\ \dfrac{\partial{L(w,b,\lambda)}}{\partial b} = \sum_{i=1}^{m}\lambda_{i}y_{i} = 0$

$L(w,b,\lambda) = \dfrac{1}{2}(\sum_{i=1}^{m}\lambda_{i}y_{i}x_{i})^{T}\sum_{j=1}^{m}\lambda_{j}y_{j}x_{j}+\sum_{i=1}^{m}\lambda_{i} - \sum_{i=1}^{m}\lambda_{i}y_{i}(w^{T}x_{i})\\ =\dfrac{1}{2}\sum_{i=1}^{m}\sum_{j=1}^{m}\lambda_{i}y_{i}\lambda_{j}y_{j}x_{i}^{T}x_{j}+\sum_{i=1}^{m}\lambda_{i} - \sum_{i=1}^{m}\sum_{j=1}^{m}\lambda_{i}y_{i}\lambda_{j}y_{j}x_{i}^{T}x_{j}\\ =\sum_{i=1}^{m}\lambda_{i} - \dfrac{1}{2}\sum_{i=1}^{m}\sum_{j=1}^{m}\lambda_{i}y_{i}\lambda_{j}y_{j}x_{i}^{T}x_{j}$

$\mathop{argmax}\limits_{\lambda}\sum_{i=1}^{m}\lambda_{i} - \dfrac{1}{2}\sum_{i=1}^{m}\sum_{j=1}^{m}\lambda_{i}y_{i}\lambda_{j}y_{j}x_{i}^{T}x_{j}, \quad s.t. \quad \sum_{i=1}^{m}\lambda_{i}y_{i} = 0$

$$$\left\{ \begin{array}{lr} \lambda_{i}>=0; & \\ y_{i}f(x_{i})-1 >=0; & \\ \lambda_{i}(y_{i}f(x_{i})-1) = 0 ; \end{array} \right.$$$

$$\lambda_{i}=0$$，则 $$y_{i}f(x_{i}) >= 1$$

$$\lambda_{i}>0$$，则 $$y_{i}f(x_{i}) = 1$$，所对应的样本刚好落在最大边界上 ( 即支持向量 )，因此训练完成后大部分的训练样本都不需要保留，最终模型仅仅与支持向量有关。

https://www.cnblogs.com/zhaozhibo/p/14925099.html