文章目錄

• • @[toc]
  • • 數(shù)據(jù)
    • • 數(shù)據(jù)集
      • 實(shí)際值
      • 估計(jì)值
    • 梯度下降算法
    • • 估計(jì)誤差
      • 代價(jià)函數(shù)
      • 學(xué)習(xí)率
      • 參數(shù)更新
    • `Python`實(shí)現(xiàn)
    • • 導(dǎo)包
      • 數(shù)據(jù)預(yù)處理
      • 迭代過(guò)程
      • 結(jié)果可視化
      • 完整代碼
    • 結(jié)果可視化
    • • 線性擬合結(jié)果
      • 代價(jià)變化

數(shù)據(jù)

數(shù)據(jù)集

$$\left(x^{(i)},y^{(i)}\right),i=1,2,?,m$$

實(shí)際值

$$y^{(i)}$$

估計(jì)值

$$h_{\theta }\left(x^{(i)}\right)={\theta }_0+{\theta }_1{x}^{(i)}$$

---

梯度下降算法

估計(jì)誤差

$$h_{\theta }\left(x^{(i)}\right)?y^{(i)}$$

代價(jià)函數(shù)

$$J(\theta )=J({\theta }_0,{\theta }_1)=\frac{1}{2m}\sum _{i=1}^m{\left(h_{\theta }\left(x^{(i)}\right)?y^{(i)}\right)}^2=\frac{1}{2m}\sum _{i=1}^m{\left({\theta }_0+{\theta }_1{x}^{(i)}?y^{(i)}\right)}^2$$

學(xué)習(xí)率

• $\alpha$是學(xué)習(xí)率，一個(gè)大于$0$的很小的經(jīng)驗(yàn)值，決定代價(jià)函數(shù)下降的程度

參數(shù)更新

$$\Delta {\theta }_j=\frac{?}{?{\theta }_j}J({\theta }_0,{\theta }_1)$$

$${\theta }_j:={\theta }_j?\alpha \Delta {\theta }_j={\theta }_j?\alpha \frac{?}{?{\theta }_j}J({\theta }_0,{\theta }_1)$$

$$
\left[
\begin{matrix}
\theta_{0} \
\theta_{1}
\end{matrix}
\right] :=

\left[
\begin{matrix}
\theta_{0} \
\theta_{1}
\end{matrix}
\right] -
\alpha

\left[
\begin{matrix}
\cfrac{\partial{J(\theta_{0} , \theta_{1})}}{\partial{\theta_{0}}} \
\cfrac{\partial{J(\theta_{0} , \theta_{1})}}{\partial{\theta_{1}}}
\end{matrix}
\right]
$$

$$\left[\begin{array}{c}\frac{?J({\theta }_0,{\theta }_1)}{?{\theta }_0}\\ \frac{?J({\theta }_0,{\theta }_1)}{?{\theta }_1}\end{array}\right]=\left[\begin{array}{c}\frac{1}{m}\sum _{i=1}^m\left(h_{\theta }\left(x^{(i)}\right)?y^{(i)}\right)\\ \frac{1}{m}\sum _{i=1}^m\left(h_{\theta }\left(x^{(i)}\right)?y^{(i)}\right)x^{(i)}\end{array}\right]=\left[\begin{array}{c}\frac{1}{m}\sum _{i=1}^m{e}^{(i)}\\ \frac{1}{m}\sum _{i=1}^m{e}^{(i)}{x}^{(i)}\end{array}\right]e^{(i)}=h_{\theta }\left(x^{(i)}\right)?y^{(i)}$$

$$\begin{array}{rl}\left[\begin{array}{c}\frac{?J({\theta }_0,{\theta }_1)}{?{\theta }_0}\\ \frac{?J({\theta }_0,{\theta }_1)}{?{\theta }_1}\end{array}\right]& =\left[\begin{array}{c}\frac{1}{m}\sum _{i=1}^m{e}^{(i)}\\ \frac{1}{m}\sum _{i=1}^m{e}^{(i)}{x}^{(i)}\end{array}\right]=\left[\begin{array}{c}\frac{1}{m}\left(e^{(1)}+e^{(2)}+?+e^{(m)}\right)\\ \frac{1}{m}\left(e^{(1)}{x}^{(1)}+e^{(2)}{x}^{(2)}+?+e^{(m)}{x}^{(m)}\right)\end{array}\right]\\ & =\frac{1}{m}\left[\begin{array}{cccc}1& 1& ?& 1\\ x^{(1)}& x^{(2)}& ?& x^{(m)}\end{array}\right]\left[\begin{array}{c}{e}^{(1)}\\ e^{(2)}\\ ?\\ e^{(m)}\end{array}\right]=\frac{1}{m}{X}^{T}e=\frac{1}{m}{X}^T(X\theta ?y)\end{array}$$

• 由上述推導(dǎo)得

$$\Delta \theta =\frac{1}{m}{X}^{T}e$$

$$\theta :=\theta ?\alpha \Delta \theta =\theta ?\alpha \frac{1}{m}{X}^{T}e$$

---

Python實(shí)現(xiàn)

導(dǎo)包

import numpy as np
import matplotlib.pyplot as plt

數(shù)據(jù)預(yù)處理

x = np.array([4, 3, 3, 4, 2, 2, 0, 1, 2, 5, 1, 2, 5, 1, 3])
y = np.array([8, 6, 6, 7, 4, 4, 2, 4, 5, 9, 3, 4, 8, 3, 6])m = len(x)x = np.c_[np.ones((m, 1)), x]
y = y.reshape(m, 1)

迭代過(guò)程

alpha = 0.01  # 學(xué)習(xí)率
iter_cnt = 1000  # 迭代次數(shù)
cost = np.zeros(iter_cnt)  # 代價(jià)數(shù)據(jù)
theta = np.zeros((2, 1))for i in range(iter_cnt):h = x.dot(theta)  # 估計(jì)值error = h - y  # 誤差值cost[i] = 1 / (2 * m) * error.T.dot(error)  # 代價(jià)值# cost[i] = 1 / (2 * m) * np.sum(np.square(error))  # 代價(jià)值# 更新參數(shù)delta_theta = 1 / m * x.T.dot(error)theta -= alpha * delta_theta

結(jié)果可視化

# 線性擬合結(jié)果
plt.scatter(x[:, 1], y, c='blue')
plt.plot(x[:, 1], h, 'r-')
plt.savefig('../pic/fit.png')
plt.show()# 代價(jià)結(jié)果
plt.plot(cost)
plt.savefig('../pic/cost.png')
plt.show()

完整代碼

import numpy as np
import matplotlib.pyplot as pltx = np.array([4, 3, 3, 4, 2, 2, 0, 1, 2, 5, 1, 2, 5, 1, 3])
y = np.array([8, 6, 6, 7, 4, 4, 2, 4, 5, 9, 3, 4, 8, 3, 6])m = len(x)x = np.c_[np.ones((m, 1)), x]
y = y.reshape(m, 1)alpha = 0.01  # 學(xué)習(xí)率
iter_cnt = 1000  # 迭代次數(shù)
cost = np.zeros(iter_cnt)  # 代價(jià)數(shù)據(jù)
theta = np.zeros((2, 1))for i in range(iter_cnt):h = x.dot(theta)  # 估計(jì)值error = h - y  # 誤差值cost[i] = 1 / (2 * m) * error.T.dot(error)  # 代價(jià)值# cost[i] = 1 / (2 * m) * np.sum(np.square(error))  # 代價(jià)值# 更新參數(shù)delta_theta = 1 / m * x.T.dot(error)theta -= alpha * delta_theta# 線性擬合結(jié)果
plt.scatter(x[:, 1], y, c='blue')
plt.plot(x[:, 1], h, 'r-')
plt.savefig('../pic/fit.png')
plt.show()# 代價(jià)結(jié)果
plt.plot(cost)
plt.savefig('../pic/cost.png')
plt.show()

---

結(jié)果可視化

線性擬合結(jié)果

代價(jià)變化

---