问题描述
我正在使用 scikit-learning 来做一些降维任务.我的训练/测试数据采用 libsvm 格式.它是一个包含 50 万列的大型稀疏矩阵.
I am using scikit-learning to do some dimension reduce task.My training/test data is in the libsvm format. It is a large sparse matrix in half million columns.
我使用load_svmlight_file函数加载数据，使用SparsePCA，scikit-learning抛出输入数据错误异常.
I use load_svmlight_file function load the data, and by using SparsePCA, the scikit-learning throw out an exception of the input data error.
如何解决?
推荐答案
稀疏 PCA 是一种用于在密集数据上寻找稀疏分解(组件具有稀疏约束)的算法.
Sparse PCA is an algorithm for finding a sparse decomposition (the components have a sparsity constraint) on dense data.
Edit: as noted in the comments sparse input for RandomizedPCA is deprecated: instead you should use sklearn.decomposition.TruncatedSVD that does precisely what RandomizedPCA used to do on sparse data but should not have been called PCA in the first place.
澄清:PCA 在数学上定义为将数据居中(去除每个特征的平均值)，然后对居中的数据应用截断的 SVD.
To clarify: PCA is mathematically defined as centering the data (removing the mean value to each feature) and then applying truncated SVD on the centered data.
由于将数据居中会破坏稀疏性并强制使用通常不再适合内存的密集表示，因此通常直接对稀疏数据进行截断 SVD(不居中).这类似于 PCA，但并不完全相同.这在 scikit-learn 中实现为 sklearn.decomposition.TruncatedSVD.
As centering the data would destroy the sparsity and force a dense representation that often does not fit in memory any more, it is common to directly do truncated SVD on sparse data (without centering). This resembles PCA but it's not exactly the same. This is implemented in scikit-learn as sklearn.decomposition.TruncatedSVD.
编辑(2019 年 3 月):正在进行对隐式居中稀疏数据实施 PCA 的工作:https://github.com/scikit-learn/scikit-learn/pull/12841
Edit (March 2019): There is ongoing work to implement PCA on sparse data with implicit centering: https://github.com/scikit-learn/scikit-learn/pull/12841
这篇关于scikit-learning 如何对 libsvm 格式的稀疏数据执行 PCA?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！