使用sklearn NMF组件重建新数据与inverse_transform不匹配

问题描述

我在训练数据上使用scikit-learn NMF模型拟合了模型.现在，我使用

I fit a model using scikit-learn NMF model on my training data. Now I perform an inverse transform of new data using

result_1 = model.inverse_transform(model.transform(new_data))

然后，我使用幻灯片15中的方程式，从NMF模型中手动获取数据的逆变换此处.

Then I compute the inverse transform of my data manually taking the components from the NMF model, using the equation as in Slide 15 here.

temp = np.dot(model.components_, model.components_.T)
transform = np.dot(np.dot(model.components_.T, np.linalg.pinv(temp)), 
model.components_)
result_2 = np.dot(new_data, transform)

我想了解为什么2个结果不匹配.在计算逆变换和重建数据时我在做什么错了?

I would like to understand why the 2 results do not match. What am I doing wrong while computing the inverse transform and reconstructing the data?

示例代码:

import numpy as np
from sklearn.decomposition import NMF

data = np.array([[0,0,1,1,1],[0,1,1,0,0],[0,1,0,0,0],[1,0,0,1,0]])
print(data)
//array([[0, 0, 1, 1, 1],
       [0, 1, 1, 0, 0],
       [0, 1, 0, 0, 0],
       [1, 0, 0, 1, 0]])


model = NMF(alpha=0.0, init='random', l1_ratio=0.0, max_iter=200, n_components=2, random_state=0, shuffle=False, solver='cd', tol=0.0001, verbose=0)
model.fit(data)
NMF(alpha=0.0, beta_loss='frobenius', init='random', l1_ratio=0.0,
  max_iter=200, n_components=2, random_state=0, shuffle=False, solver='cd',
  tol=0.0001, verbose=0)

new_data = np.array([[0,0,1,0,0], [1,0,0,0,0]])
print(new_data)
//array([[0, 0, 1, 0, 0],
       [1, 0, 0, 0, 0]])

result_1 = model.inverse_transform(model.transform(new_data))
print(result_1)
//array([[ 0.09232497,  0.38903892,  0.36668712,  0.23067627,  0.1383513 ],
       [ 0.0877082 ,  0.        ,  0.12131779,  0.21914115,  0.13143295]])

temp = np.dot(model.components_, model.components_.T)
transform = np.dot(np.dot(model.components_.T, np.linalg.pinv(temp)), model.components_)
result_2 = np.dot(new_data, transform)
print(result_2)
//array([[ 0.09232484,  0.389039  ,  0.36668699,  0.23067595,  0.13835111],
       [ 0.09193481, -0.05671439,  0.09232484,  0.22970145,  0.13776664]])

注意:尽管这不是描述我的问题的最佳数据，但代码本质上是相同的.在实际情况下，result_1和result_2彼此也有很大不同. data和new_data也是大数组.

Note: Although this is not the best data describing my issue, the code is essentially the same. Also result_1 and result_2 are much more different from each other in the actual case. data and new_data are also large arrays.

推荐答案

会发生什么

在scikit-learn中，NMF不仅仅是简单的矩阵乘法:它可以优化！

解码(inverse_transform)是线性的:模型计算X_decoded = dot(W, H)，其中W是编码矩阵，而H=model.components_是模型参数的学习矩阵.

Decoding (inverse_transform) is linear: the model calculates X_decoded = dot(W, H), where W is the encoded matrix, and H=model.components_ is a learned matrix of model parameters.

编码(transform)是非线性的:它执行W = argmin(loss(X_original, H, W))(仅针对W)，其中损失是X_original和，再加上一些额外的惩罚(W的L1和L2范数)，并以W必须为非负数为约束.最小化是通过坐标下降执行的，结果在X_original中可能是非线性的.因此，不能简单地通过将矩阵相乘得到W.

Encoding (transform), however, is nonlinear : it performs W = argmin(loss(X_original, H, W)) (with respect to W only), where loss is mean squared error between X_original and dot(W, H), plus some additional penalties (L1 and L2 norms of W), and with the constraint that W must be non-negative. Minimization is performed by coordinate descent, and result may be nonlinear in X_original. Thus, you cannot simply get W by multiplying matrices.

NMF必须执行这种奇怪的计算，因为否则，该模型可能会产生负面结果.确实，在您自己的示例中，您可以尝试通过矩阵乘法

NMF has to perform such strange calculations because, otherwise, the model may produce negative results. Indeed, in your own example, you could try to perform transform by matrix multiplication

 print(np.dot(new_data, np.dot(model.components_.T, np.linalg.pinv(temp))))

并得到包含负数的结果W:

and get the result W that contains negative numbers:

[[ 0.17328927  0.39649966]
 [ 0.1725572  -0.05780202]]

但是，NMF中的坐标下降通过稍微修改矩阵避免了这个问题:

However, the coordinate descent within NMF avoids this problem by slightly modifying the matrix:

 print(model.transform(new_data))

给出非负结果

[[0.17328951 0.39649958]
 [0.16462405 0.        ]]

您会发现它不只是从下面裁剪W矩阵，而且还修改了正元素，以提高拟合度(并遵守正则化惩罚).

You can see that it does not simply clip W matrix from below, but modifies the positive elements as well, in order to improve the fit (and obey the regularization penalties).

这篇关于使用sklearn NMF组件重建新数据与inverse_transform不匹配的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！