scikit learn的归责变压器的docs表示
当轴＝0时，仅在拟合中包含缺失值的列在变换时被丢弃。
由于插补器返回一个numpy数组，我如何检查哪些特征在插补期间被丢弃，或者相应地，哪些特征在插补后被保留？
下面是一个简单的例子：
import pandas as pd
import numpy as np
from sklearn.preprocessing import Imputer
df = pd.DataFrame(np.random.randn(10, 5), columns=['a', 'b', 'c', 'd', 'e'])
df['f'] = len(df3)*['NaN']
这是数据框：
>>> df
a b c d e f
0 -1.284658 0.246541 -1.120987 0.559911 -1.189870 NaN
1 0.773717 0.430597 -0.004346 -1.292080 1.993266 NaN
2 1.418761 -0.004749 -0.181932 -0.305756 -0.135870 NaN
3 0.418673 -0.376318 -0.860783 0.074135 -1.034095 NaN
4 -0.019873 0.006210 0.364384 1.029895 -0.188727 NaN
5 0.903661 0.123575 -0.556970 1.344985 -1.109806 NaN
6 -0.069168 -0.385597 0.684345 0.645920 1.159898 NaN
7 0.695782 0.030239 -0.777304 -0.037102 2.053028 NaN
8 -0.256409 0.106735 -0.729710 0.254626 1.064925 NaN
9 0.235507 -0.087767 0.626121 1.391286 0.449158 NaN
现在我创建一个输入：
imp = Imputer(missing_values='NaN', strategy='mean', axis=0)
imp.fit(df)
imputed = imp.transform(df)
这是输入返回的numpy数组。
>>> imputed
array([[-1.28465763, 0.24654083, -1.12098675, 0.55991059, -1.18986998],
[ 0.77371694, 0.43059674, -0.0043461 , -1.29208032, 1.99326594],
[ 1.41876145, -0.0047488 , -0.18193164, -0.30575631, -0.13586974],
[ 0.41867326, -0.37631792, -0.86078293, 0.07413458, -1.03409532],
最佳答案
如何检查哪些特征在插补期间被丢弃？
包含所有NaNs的列将被丢弃。您可以在不使用fit执行transform和df.isnull().all()过程的情况下检查此项。其中True，这些是将被丢弃的“功能”。
不过，确切的答案是将verbose=1添加到您的输入中，如下所示：
imp = Imputer(verbose=1)
为了使这个示例更清楚地了解发生了什么，请在
df中添加另一个包含所有NaN的列。df.insert(2, 'g', np.nan)
df现在看起来是这样的： a b g c d e f
0 -1.284658 0.246541 NaN -1.120987 0.559911 -1.189870 NaN
1 0.773717 0.430597 NaN -0.004346 -1.292080 1.993266 NaN
2 1.418761 -0.004749 NaN -0.181932 -0.305756 -0.135870 NaN
3 0.418673 -0.376318 NaN -0.860783 0.074135 -1.034095 NaN
4 -0.019873 0.006210 NaN 0.364384 1.029895 -0.188727 NaN
5 0.903661 0.123575 NaN -0.556970 1.344985 -1.109806 NaN
6 -0.069168 -0.385597 NaN 0.684345 0.645920 1.159898 NaN
7 0.695782 0.030239 NaN -0.777304 -0.037102 2.053028 NaN
8 -0.256409 0.106735 NaN -0.729710 0.254626 1.064925 NaN
9 0.235507 -0.087767 NaN 0.626121 1.391286 0.449158 NaN
正在运行。。。
imp.fit(df)
imp.transform(df)
现在输出以下“verbose”消息，告诉您哪些列已被删除：
Warning (from warnings module):
File "C:\Python34\lib\site-packages\sklearn\preprocessing\imputation.py", line 347
"observed values: %s" % missing)
UserWarning: Deleting features without observed values: [2 6]
array([[-1.284658, 0.246541, -1.120987, 0.559911, -1.18987 ],
[ 0.773717, 0.430597, -0.004346, -1.29208 , 1.993266],
[ 1.418761, -0.004749, -0.181932, -0.305756, -0.13587 ],
[ 0.418673, -0.376318, -0.860783, 0.074135, -1.034095],
[-0.019873, 0.00621 , 0.364384, 1.029895, -0.188727],
[ 0.903661, 0.123575, -0.55697 , 1.344985, -1.109806],
[-0.069168, -0.385597, 0.684345, 0.64592 , 1.159898],
[ 0.695782, 0.030239, -0.777304, -0.037102, 2.053028],
[-0.256409, 0.106735, -0.72971 , 0.254626, 1.064925],
[ 0.235507, -0.087767, 0.626121, 1.391286, 0.449158]])
哪些特征在插补后保留了下来？
插补后保留的列和值。
使用我以前的
[2 6]，如果我们将一些df添加到组合中：df.loc[[1, 7, 3], ['a', 'c', 'e']] = np.nan
NaN如下所示： a b g c d e f
0 -1.284658 0.246541 NaN -1.120987 0.559911 -1.189870 NaN
1 NaN 0.430597 NaN NaN -1.292080 NaN NaN
2 1.418761 -0.004749 NaN -0.181932 -0.305756 -0.135870 NaN
3 NaN -0.376318 NaN NaN 0.074135 NaN NaN
4 -0.019873 0.006210 NaN 0.364384 1.029895 -0.188727 NaN
5 0.903661 0.123575 NaN -0.556970 1.344985 -1.109806 NaN
6 -0.069168 -0.385597 NaN 0.684345 0.645920 1.159898 NaN
7 NaN 0.030239 NaN NaN -0.037102 NaN NaN
8 -0.256409 0.106735 NaN -0.729710 0.254626 1.064925 NaN
9 0.235507 -0.087767 NaN 0.626121 1.391286 0.449158 NaN
重要的是要明白你在使用什么样的归责策略。
df的默认值是mean。这意味着它将用给定列的平均值替换Imputer值。要证明这一点，请先检查每一列的平均值：
>>> df.mean()
a 0.132546
b 0.008947
g NaN
c -0.130678
d 0.366582
e 0.007101
f NaN
dtype: float64
然后，您可以进行拟合和转换，看看转换后的输入数据中是否有任何值位于
NaN超参数中。imp = Imputer(verbose=1)
imp.fit(df)
imp.transform(df)
返回以下结果-同样，需要注意的关键是
imp.statistics_值已替换为给定列的NaN。例如，无论您在第一列中看到mean处，您都会注意到它们出现在第1、3和7行（以前0.13254586s）：Warning (from warnings module):
File "C:\Python34\lib\site-packages\sklearn\preprocessing\imputation.py", line 347
"observed values: %s" % missing)
UserWarning: Deleting features without observed values: [2 6]
array([[-1.284658 , 0.246541 , -1.120987 , 0.559911 , -1.18987 ],
[ 0.13254586, 0.430597 , -0.13067843, -1.29208 , 0.00710114],
[ 1.418761 , -0.004749 , -0.181932 , -0.305756 , -0.13587 ],
[ 0.13254586, -0.376318 , -0.13067843, 0.074135 , 0.00710114],
[-0.019873 , 0.00621 , 0.364384 , 1.029895 , -0.188727 ],
[ 0.903661 , 0.123575 , -0.55697 , 1.344985 , -1.109806 ],
[-0.069168 , -0.385597 , 0.684345 , 0.64592 , 1.159898 ],
[ 0.13254586, 0.030239 , -0.13067843, -0.037102 , 0.00710114],
[-0.256409 , 0.106735 , -0.72971 , 0.254626 , 1.064925 ],
[ 0.235507 , -0.087767 , 0.626121 , 1.391286 , 0.449158 ]])
如果你想做一个布尔比较，看看是什么值被估算，你可以做以下工作（不是万无一失，而是一个最可靠的方法）：
np.reshape(np.in1d(imp.transform(df), imp.statistics_), imp.transform(df).shape)
array([[False, False, False, False, False],
[ True, False, True, False, True],
[False, False, False, False, False],
[ True, False, True, False, True],
[False, False, False, False, False],
[False, False, False, False, False],
[False, False, False, False, False],
[ True, False, True, False, True],
[False, False, False, False, False],
[False, False, False, False, False]], dtype=bool)
关于python - 检查scikitlearn不当丢弃的功能，我们在Stack Overflow上找到一个类似的问题：https://stackoverflow.com/questions/38277014/