问题描述
这是一个简单的代码工作实现，我在 Python 的 scikit-learn 中使用高斯过程回归 (GPR) 和二维输入(即 x1 和 x2 上的网格code>) 和一维输出 (y).
Here is a simple working implementation of a code where I use Gaussian process regression (GPR) in Python's scikit-learn with 2-dimensional inputs (i.e grid over x1 and x2) and 1-dimensional outputs (y).
import numpy as np
from matplotlib import pyplot as plt
from sklearn.gaussian_process import GaussianProcessRegressor
from sklearn.gaussian_process.kernels import RBF, ConstantKernel as C
from mpl_toolkits.mplot3d import Axes3D
# Example independent variable (observations)
X = np.array([[0.,0.], [1.,0.], [2.,0.], [3.,0.], [4.,0.],
[5.,0.], [6.,0.], [7.,0.], [8.,0.], [9.,0.], [10.,0.],
[11.,0.], [12.,0.], [13.,0.], [14.,0.],
[0.,1.], [1.,1.], [2.,1.], [3.,1.], [4.,1.],
[5.,1.], [6.,1.], [7.,1.], [8.,1.], [9.,1.], [10.,1.],
[11.,1.], [12.,1.], [13.,1.], [14.,1.],
[0.,2.], [1.,2.], [2.,2.], [3.,2.], [4.,2.],
[5.,2.], [6.,2.], [7.,2.], [8.,2.], [9.,2.], [10.,2.],
[11.,2.], [12.,2.], [13.,2.], [14.,2.]])#.T
# Example dependent variable (observations) - noiseless case
y = np.array([4.0, 3.98, 4.01, 3.95, 3.9, 3.84,3.8,
3.73, 2.7, 1.64, 0.62, 0.59, 0.3,
0.1, 0.1,
4.4, 3.9, 4.05, 3.9, 3.5, 3.4,3.3,
3.23, 2.6, 1.6, 0.6, 0.5, 0.32,
0.05, 0.02,
4.0, 3.86, 3.88, 3.76, 3.6, 3.4,3.2,
3.13, 2.5, 1.6, 0.55, 0.51, 0.23,
0.11, 0.01])
x1 = np.linspace(0, 14, 20)
x2 = np.linspace(0, 5, 100)
i = 0
inputs_x = []
while i < len(x1):
j = 0
while j < len(x2):
inputs_x.append([x1[i],x2[j]])
j = j + 1
i = i + 1
inputs_x_array = np.array(inputs_x)
# Instantiate a Gaussian Process model
kernel = C(1.0, (1e-3, 1e3)) * RBF((1e-2, 1e2), (1e-2, 1e2))
gp = GaussianProcessRegressor(kernel=kernel, n_restarts_optimizer=20)
gp.fit(X, y.reshape(-1,1)) #removing reshape results in a different error
y_pred, sigma = gp.predict(inputs_x_array, return_std=True)
它有效，但是在定义内核时，我如何确保为不同的输入(即 x1 和 x2)设置不同的超参数(例如不同的尺度长度)?在上面的示例中，使用的标准内核是径向基函数 (RBF)，尽管有两个输入维度，但它似乎只有一个长度尺度.但是如何训练这个内核(或自定义内核，例如双曲正切)来解释不同输入维度的不同超参数?
It works, but when defining the kernel, how can I ensure I set different hyperparameters (e.g. different scale lengths) for my different inputs (i.e. x1 and x2)? In the example above, the standard kernel used is a radial basis function (RBF) which appears to have a single length scale despite two input dimensions. But how could this kernel (or a custom kernel, e.g. hyperbolic tangent) be trained to account for different hyperparameters for the different input dimensions?
推荐答案
您将需要各向异性内核，目前 sklearn 中只有少数内核支持这些内核.RBF 就是这样一个示例，您可以在其中提供一个列表作为 length_scale 参数的输入.例如，RBF(length_scale = [1, 10], length_scale_bounds=(1e-5, 1e5)) 是完全有效的，其中 1 代表 x1，10 代表x2.
You'll need anisotropic kernels, which are only supported by a few kernels in sklearn for the moment. RBF is such an example where you can give a list as input for the length_scale parameter. For example, RBF(length_scale = [1, 10], length_scale_bounds=(1e-5, 1e5)) is perfectly valid, where 1 holds for x1 and 10 holds for x2.
然而，sklearn 中的大多数内核都是各向同性的，目前不支持各向异性情况.如果你想要更多的自由，我建议你看看其他的包(比如 GPy)或者你总是可以尝试实现你自己的各向异性内核.
Most kernels in sklearn however are isotropic, where the anisotropic case is -currently- not supported. If you want more freedom, I suggest you take a look at other packages (like GPy) or you can always try to implement your own anisotropic kernel.
这篇关于训练多维高斯过程回归的超参数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！