逻辑回归（Logistic Regression）详解

1. 介绍

逻辑回归是一种用于解决分类问题的统计学习方法。它适用于二分类问题，即将样本分为两个类别。逻辑回归的主要思想是通过一个函数来预测输入的样本属于某个类别的概率。在本文中，我们将深入探讨逻辑回归的原理、实现方法以及使用Python和JavaScript编写的示例代码。
逻辑回归（Logistic Regression）详解-LMLPHP

2. 原理

逻辑回归的核心是sigmoid函数，也称为logistic函数，其公式如下：

σ ( z ) = 1 1 + e − z \sigma(z) = \frac{1}{1 + e^{-z}} σ(z)=1+e−z1

其中，z是输入样本的线性组合，即：

z = w 0 + w 1 x 1 + w 2 x 2 + . . . + w n x n z = w_0 + w_1x_1 + w_2x_2 + ... + w_nx_n z=w0+w1x1+w2x2+...+wnxn

其中 w 0 , w 1 , . . . , w n w_0, w_1, ..., w_n w0,w1,...,wn是模型的参数， x 1 , x 2 , . . . , x n x_1, x_2, ..., x_n x1,x2,...,xn是输入样本的特征值。sigmoid函数将z映射到[0, 1]之间的值，表示样本属于正类的概率。

3. 实现方法

3.1 参数学习

逻辑回归的参数学习通常使用最大似然估计（Maximum Likelihood Estimation, MLE）方法。给定训练样本 ( x 1 , y 1 ) , ( x 2 , y 2 ) , . . . , ( x m , y m ) (x_1, y_1), (x_2, y_2), ..., (x_m, y_m) (x1,y1),(x2,y2),...,(xm,ym)，其中 x i x_i xi是样本特征向量， y i y_i yi是类别标签（0或1）。模型的参数可以通过最大化似然函数来学习，即：

max ⁡ w ∏ i = 1 m P ( y i ∣ x i ; w ) \max_{w} \prod_{i=1}^{m} P(y_i|x_i; w) wmaxi=1∏mP(yi∣xi;w)

3.2 损失函数

逻辑回归的常用损失函数是交叉熵损失函数（Cross Entropy Loss），其公式如下：

J ( w ) = − 1 m ∑ i = 1 m [ y i log ⁡ ( y ^ i ) + ( 1 − y i ) log ⁡ ( 1 − y ^ i ) ] J(w) = -\frac{1}{m} \sum_{i=1}^{m} [y_i\log(\hat{y}_i) + (1-y_i)\log(1-\hat{y}_i)] J(w)=−m1i=1∑m[yilog(y^i)+(1−yi)log(1−y^i)]

其中， y ^ i \hat{y}_i y^i是模型对样本 x i x_i xi的预测值。

4. 示例代码

4.1 Python示例代码

import numpy as np

class LogisticRegression:
    def __init__(self, learning_rate=0.01, num_iterations=1000):
        self.learning_rate = learning_rate
        self.num_iterations = num_iterations
        self.weights = None
        self.bias = None

    def sigmoid(self, z):
        return 1 / (1 + np.exp(-z))

    def fit(self, X, y):
        m, n = X.shape
        self.weights = np.zeros(n)
        self.bias = 0

        for _ in range(self.num_iterations):
            z = np.dot(X, self.weights) + self.bias
            y_pred = self.sigmoid(z)

            dw = (1 / m) * np.dot(X.T, (y_pred - y))
            db = (1 / m) * np.sum(y_pred - y)

            self.weights -= self.learning_rate * dw
            self.bias -= self.learning_rate * db

    def predict(self, X):
        z = np.dot(X, self.weights) + self.bias
        y_pred = self.sigmoid(z)
        return np.round(y_pred)

# 示例用法
X_train = np.array([[1, 2], [2, 3], [3, 4], [4, 5]])
y_train = np.array([0, 0, 1, 1])

model = LogisticRegression()
model.fit(X_train, y_train)

X_test = np.array([[5, 6], [6, 7]])
print(model.predict(X_test))  # 输出预测结果

4.2 JavaScript示例代码

class LogisticRegression {
    constructor(learningRate = 0.01, numIterations = 1000) {
        this.learningRate = learningRate;
        this.numIterations = numIterations;
        this.weights = null;
        this.bias = null;
    }

    sigmoid(z) {
        return 1 / (1 + Math.exp(-z));
    }

    fit(X, y) {
        const m = X.length;
        const n = X[0].length;
        this.weights = new Array(n).fill(0);
        this.bias = 0;

        for (let i = 0; i < this.numIterations; i++) {
            let z = 0;
            for (let j = 0; j < m; j++) {
                z += X[j].reduce((acc, val, idx) => acc + val * this.weights[idx], 0) + this.bias;
            }
            const yPred = this.sigmoid(z);

            let dw = new Array(n).fill(0);
            let db = 0;
            for (let j = 0; j < m; j++) {
                const diff = yPred - y[j];
                for (let k = 0; k < n; k++) {
                    dw[k] += X[j][k] * diff;
                }
                db += diff;
            }
            dw = dw.map(val => val / m);
            db /= m;

            for (let j = 0; j < n; j++) {
                this.weights[j] -= this.learningRate * dw[j];
            }
            this.bias -= this.learningRate * db;
        }
    }

    predict(X) {
        const m = X.length;
        const predictions = [];
        for (let i = 0; i < m; i++) {
            let z = 0;
            for (let j = 0; j < X[i].length; j++) {
                z += X[i][j] * this.weights[j];
            }
            z += this.bias;
            const yPred = this.sigmoid(z);
            predictions.push(Math.round(yPred));
        }
        return predictions;
    }
}

// 示例用法
const XTrain = [[1, 2], [2, 3], [3, 4], [4, 5]];
const yTrain = [0, 0, 1, 1];

const model = new LogisticRegression();
model.fit(XTrain, yTrain);

const XTest = [[5, 6], [6, 7]];
console.log(model.predict(XTest)); // 输出预测结果

5. 结论

通过逻辑回归模型，我们可以根据学生的学习时间来预测其是否会及格考试。逻辑回归是一种简单而有效的分类算法，在实际应用中具有广泛的用途。通过深入理解逻辑回归，我们可以更好地应用它解决实际问题。

平底斜