似然函数

 原理：极大似然估计是建立在极大似然原理的基础上的一个统计方法，是概率论在统计学中的应用。极大似然估计提供了一种给定观察数据来评估模型参数的方法，即：“模型已定，参数未知”。通过若干次试验，观察其结果，利用试验结果得到某个参数值能够使样本出现的概率为最大，则称为极大似然估计。

    由于样本集中的样本都是独立同分布，可以只考虑一类样本集D，来估计参数向量θ。记已知的样本集为：

 似然函数（linkehood function）：联合概率密度函数称为相对于的θ的似然函数。

对于似然函数的定义有些不正确，只看求导过程的推导

 

 

In [7]:

import numpy as npimport matplotlib.pyplot as pltnp.random.seed(666)X = np.random.normal(0, 1, size=(200, 2))y = np.array(X[:, 0] ** 2 + X[:, 1] < 1.5, dtype='int')for _ in range(20):    y[np.random.randint(200)] = 1  # 生成噪音数据plt.scatter(X[y == 0, 0], X[y == 0, 1])plt.scatter(X[y == 1, 0], X[y == 1, 1])plt.show()

 

In [25]:

from sklearn.model_selection import train_test_splitX_train, X_test, y_train, y_test = train_test_split(X, y, random_state=666)

In [26]:

from sklearn.linear_model import LogisticRegressionfrom sklearn.preprocessing import PolynomialFeaturesfrom sklearn.pipeline import Pipelinelog_reg = LogisticRegression(solver='lbfgs')log_reg.fit(X_train, y_train)

Out[26]:

LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,          intercept_scaling=1, max_iter=100, multi_class='warn',          n_jobs=None, penalty='l2', random_state=None, solver='lbfgs',          tol=0.0001, verbose=0, warm_start=False)

In [27]:

log_reg.score(X_train, y_train)

Out[27]:

0.7933333333333333

In [28]:

log_reg.score(X_test, y_test)

Out[28]:

0.86

In [34]:

def plot_decision_boundary(model, axis):    x0, x1 = np.meshgrid(        np.linspace(axis[0], axis[1], int((axis[1] - axis[0]) * 100)),        np.linspace(axis[2], axis[3], int((axis[3] - axis[2]) * 100))    )    X_new = np.c_[x0.ravel(), x1.ravel()]    y_predict = model.predict(X_new)    zz = y_predict.reshape(x0.shape)    from matplotlib.colors import ListedColormap    custom_cmap = ListedColormap(['#EF9A9A', '#FFF59D', '#90CAF9'])    plt.contourf(x0, x1, zz, cmap=custom_cmap)

In [35]:

plot_decision_boundary(log_reg,axis=[-4,4,-4,4])plt.scatter(X[y == 0, 0], X[y == 0, 1])plt.scatter(X[y == 1, 0], X[y == 1, 1])plt.show()

 

 

多项式特征应用于逻辑回归

In [38]:

from sklearn.preprocessing import StandardScalerdef PolynomialLogisticRegression(degree):    return Pipeline([        ('Poly', PolynomialFeatures(degree=degree)),        ('std_scaler', StandardScaler()),        ('Logistic', LogisticRegression(solver='lbfgs'))    ])log_reg2 = PolynomialLogisticRegression(2)log_reg2.fit(X_train, y_train)log_reg2.score(X_train, y_train)

Out[38]:

0.9066666666666666

In [39]:

log_reg2.score(X_test, y_test)

Out[39]:

0.94

In [40]:

plot_decision_boundary(log_reg2, axis=[-4, 4, -4, 4])plt.scatter(X[y == 0, 0], X[y == 0, 1])plt.scatter(X[y == 1, 0], X[y == 1, 1])plt.show()