Skip to content

Latest commit

 

History

History
40 lines (37 loc) · 1.69 KB

14.1.md

File metadata and controls

40 lines (37 loc) · 1.69 KB

极大似然函数期望$E(LL)$如下式: $$ \begin{aligned} E(LL)&=\sum_{(x_j,y_j)\in D_l}\ln p(x_j,y_j)+\sum_{x_j\in D_u}\sum_i\gamma_{ji}\ln p(x_j,y_j=i)\ &=\sum_{(x_j,y_j)\in D_l}\ln[\alpha_{y_j}\Bbb{N}(x_j|\mu_{y_j},\Sigma_{y_j})] +\sum_{x_j\in D_u}\sum_i\gamma_{ji}\ln[\alpha_i\Bbb{N}(x_j|\mu_i,\Sigma_i)]\ \end{aligned} $$ 多维的高斯分布的似然函数对均值矩阵协方差矩阵求导为: $$ \begin{aligned} &\Bbb{N}(x_j|\mu_i,\Sigma_i)=\frac{1}{(2\pi)^{n/2}|\Sigma_i|^{1/2}}exp[-\frac{1}{2}(x_j-\mu_i)^T\Sigma_i^{-1}(x_j-\mu_i)]\ &\nabla_{\mu_i}\ln\Bbb{N}(x_j|\mu_i,\Sigma_i)=\Sigma_i^{-1}(x_j-\mu_i)\ &\nabla_{\Sigma_i^{-1}}\ln\Bbb{N}(x_j|\mu_i,\Sigma_i)=\frac{1}{2}[\Sigma_i-(x_j-\mu_i)(x_j-\mu_i)^T] \end{aligned} $$ 最大化似然函数,并对均值矩阵协方差矩阵求导,令其为0即可得$mu_i$和$\Sigma_i$:

​ $$ \begin{aligned} &\bf\nabla_{\mu_i}E(LL)\ &=\sum_{(x_j,y_j)\in D_l}I(y_j=i)\Sigma_{i}^{-1}(x_j-\mu_i)+\sum_{x_j\in D_u}\gamma_{ji}\Sigma_{i}^{-1}(x_j-\mu_i)]\ &=0\ &\Rightarrow \mu_i=\frac{1}{l_i+\sum_{D_u}\gamma_{ji}}[\sum_{(x_j,y_j)\in D_l}I(y_j=i)x_j+\sum_{x_j\in D_u}\gamma_{ji},x_j] \end{aligned} $$

$$ \begin{aligned} &\bf\nabla_{\Sigma_i^{-1}}E(LL)\\ &=\sum_{(x_j,y_j)\in D_l}I(y_j=i)\frac{1}{2}[\Sigma_i-(x_j-\mu_i)(x_j-\mu_i)^T]\\ &\quad+\sum_{x_j\in D_u}\gamma_{ji}\frac{1}{2}[\Sigma_i-(x_j-\mu_i)(x_j-\mu_i)^T\\ &=0\\ &\Rightarrow \Sigma_i=\frac{1}{l_i+\sum_{D_u}\gamma_{ji}}[\sum_{(x_j,y_j)\in D_l}I(y_j=i)(x_j-\mu_i)(x_j-\mu_i)^T\\ &\qquad\qquad+\sum_{x_j\in D_u}\gamma_{ji},(x_j-\mu_i)(x_j-\mu_i)^T] \end{aligned} $$

${\alpha_i}$可采用拉格朗日方法求解约束问题。