问题描述
我想将缩放(使用 sklearn.preprocessing 中的 StandardScaler())应用于 Pandas 数据帧.以下代码返回一个 numpy 数组,因此我丢失了所有列名和索引.这不是我想要的.
I want to apply scaling (using StandardScaler() from sklearn.preprocessing) to a pandas dataframe. The following code returns a numpy array, so I lose all the column names and indeces. This is not what I want.
features = df[["col1", "col2", "col3", "col4"]]
autoscaler = StandardScaler()
features = autoscaler.fit_transform(features)
我在网上找到的一个解决方案"是:
A "solution" I found online is:
features = features.apply(lambda x: autoscaler.fit_transform(x))
它似乎有效,但会导致弃用警告:
It appears to work, but leads to a deprecationwarning:
/usr/lib/python3.5/site-packages/sklearn/preprocessing/data.py:583:弃用警告:在 0.17 中不推荐将一维数组作为数据传递并将在 0.19 中引发 ValueError.使用X.reshape(-1, 1) 如果您的数据具有单个特征或 X.reshape(1, -1)如果它包含单个样本.
因此我尝试了:
features = features.apply(lambda x: autoscaler.fit_transform(x.reshape(-1, 1)))
但这给出了:
回溯(最近一次调用最后一次):文件./analysis.py",第 91 行,在features = features.apply(lambda x: autoscaler.fit_transform(x.reshape(-1, 1))) 文件/usr/lib/python3.5/site-packages/pandas/core/frame.py",第 3972 行,在申请返回 self._apply_standard(f, axis, reduce=reduce) 文件/usr/lib/python3.5/site-packages/pandas/core/frame.py",第 4081 行,在_apply_standardresult = self._constructor(data=results, index=index) 文件/usr/lib/python3.5/site-packages/pandas/core/frame.py",第226行,在初始化mgr = self._init_dict(data, index, columns, dtype=dtype) 文件/usr/lib/python3.5/site-packages/pandas/core/frame.py",第 363 行,在_init_dictdtype=dtype) 文件/usr/lib/python3.5/site-packages/pandas/core/frame.py",第 5163 行,在_arrays_to_mgr数组 = _homogenize(arrays, index, dtype) 文件/usr/lib/python3.5/site-packages/pandas/core/frame.py",第 5477 行,在_同质化raise_cast_failure=False) 文件/usr/lib/python3.5/site-packages/pandas/core/series.py",第 2885 行,在 _sanitize_array 中raise Exception('数据必须是一维的') 异常:数据必须是一维的
如何对 Pandas 数据框应用缩放,同时保持数据框完好无损?尽可能不复制数据.
How do I apply scaling to the pandas dataframe, leaving the dataframe intact? Without copying the data if possible.
推荐答案
您可以使用 as_matrix()
.随机数据集示例:
You could convert the DataFrame as a numpy array using as_matrix()
. Example on a random dataset:
根据上述 as_matrix()
文档的最后一句,将 as_matrix()
更改为 values
,(它不会更改结果):
Changing as_matrix()
to values
, (it doesn't change the result) per the last sentence of the as_matrix()
docs above:
一般建议使用‘.values’.
import pandas as pd
import numpy as np #for the random integer example
df = pd.DataFrame(np.random.randint(0.0,100.0,size=(10,4)),
index=range(10,20),
columns=['col1','col2','col3','col4'],
dtype='float64')
注意,索引是 10-19:
Note, indices are 10-19:
In [14]: df.head(3)
Out[14]:
col1 col2 col3 col4
10 3 38 86 65
11 98 3 66 68
12 88 46 35 68
现在fit_transform
DataFrame得到scaled_features
array
:
Now fit_transform
the DataFrame to get the scaled_features
array
:
from sklearn.preprocessing import StandardScaler
scaled_features = StandardScaler().fit_transform(df.values)
In [15]: scaled_features[:3,:] #lost the indices
Out[15]:
array([[-1.89007341, 0.05636005, 1.74514417, 0.46669562],
[ 1.26558518, -1.35264122, 0.82178747, 0.59282958],
[ 0.93341059, 0.37841748, -0.60941542, 0.59282958]])
将缩放后的数据分配给 DataFrame(注意:使用 index
和 columns
关键字参数来保留原始索引和列名:
Assign the scaled data to a DataFrame (Note: use the index
and columns
keyword arguments to keep your original indices and column names:
scaled_features_df = pd.DataFrame(scaled_features, index=df.index, columns=df.columns)
In [17]: scaled_features_df.head(3)
Out[17]:
col1 col2 col3 col4
10 -1.890073 0.056360 1.745144 0.466696
11 1.265585 -1.352641 0.821787 0.592830
12 0.933411 0.378417 -0.609415 0.592830
编辑 2:
遇到了 sklearn-pandas 包.它专注于使 scikit-learn 更易于与 Pandas 一起使用.sklearn-pandas
在您需要对 DataFrame
的列子集应用多种类型的转换时特别有用,这是一种更常见的场景.它已记录在案,但这就是您实现我们刚刚执行的转换的方式.
Came across the sklearn-pandas package. It's focused on making scikit-learn easier to use with pandas. sklearn-pandas
is especially useful when you need to apply more than one type of transformation to column subsets of the DataFrame
, a more common scenario. It's documented, but this is how you'd achieve the transformation we just performed.
from sklearn_pandas import DataFrameMapper
mapper = DataFrameMapper([(df.columns, StandardScaler())])
scaled_features = mapper.fit_transform(df.copy(), 4)
scaled_features_df = pd.DataFrame(scaled_features, index=df.index, columns=df.columns)
这篇关于如何将 sklearn fit_transform 与 pandas 一起使用并返回数据帧而不是 numpy 数组?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!