问题描述
我通过scikit-learn库学习机器学习.我应用决策树分类器和随机森林分类器通过以下代码对我的数据进行处理:
I learn the machine learning with the scikit-learn library. I apply thedecision tree classifier and the random forest classifier to my data with this code:
def decision_tree(train_X, train_Y, test_X, test_Y):
clf = tree.DecisionTreeClassifier()
clf.fit(train_X, train_Y)
return clf.score(test_X, test_Y)
def random_forest(train_X, train_Y, test_X, test_Y):
clf = RandomForestClassifier(n_estimators=1)
clf = clf.fit(X, Y)
return clf.score(test_X, test_Y)
为什么对于随机森林分类器而言，结果要好得多(对于100次运行，随机采样2/3的训练数据和1/3的测试数据)?
Why the result are so much better for the random forest classifier (for 100 runs, with randomly sampling 2/3 of data for the training and 1/3 for the test)?
100%|███████████████████████████████████████| 100/100 [00:01<00:00, 73.59it/s] Algorithm: Decision Tree Min : 0.3883495145631068 Max : 0.6476190476190476 Mean : 0.4861783113770316 Median : 0.48868030937802126 Stdev : 0.047158171852401135 Variance: 0.0022238931724605985 100%|███████████████████████████████████████| 100/100 [00:01<00:00, 85.38it/s] Algorithm: Random Forest Min : 0.6846846846846847 Max : 0.8653846153846154 Mean : 0.7894823428836184 Median : 0.7906101571063208 Stdev : 0.03231671150915106 Variance: 0.0010443698427656967
具有一个估计量的随机森林估计量不仅仅是决策树吗?我做错了什么或误解了这个概念吗?
The random forest estimators with one estimator isn't just a decision tree?Have i done something wrong or misunderstood the concept?
感谢您的答复.
推荐答案
嗯，这是一个很好的问题，答案是否；随机森林算法不只是一袋简单的独立生长的决策树.
Well, this is a good question, and the answer turns out to be no; the Random Forest algorithm is more than a simple bag of individually-grown decision trees.
除了由于合并许多树而引起的随机性外，随机森林(RF)算法还以两种不同的方式在构建个体树时合并了随机性，而简单决策树中都没有这种随机性( DT)算法.
Apart from the randomness induced from ensembling many trees, the Random Forest (RF) algorithm also incorporates randomness when building individual trees in two distinct ways, none of which is present in the simple Decision Tree (DT) algorithm.
第一个是在每个树节点上寻找最佳分割时要考虑的特征数量:DT会考虑所有特征，RF会考虑它们的随机子集，其大小等于参数max_features(请参见文档).
The first is the number of features to consider when looking for the best split at each tree node: while DT considers all the features, RF considers a random subset of them, of size equal to the parameter max_features (see the docs).
第二个是，虽然DT考虑了整个训练集，但单个RF树仅考虑了它的一个自举子样本；再次从 docs 中
The second is that, while DT considers the whole training set, a single RF tree considers only a bootstrapped sub-sample of it; from the docs again:
RF算法本质上是两个独立思想的组合:装袋和特征随机选择(请参见 Wikipedia条目以获得很好的概述).套袋本质上是我的第二点，但适用于合奏.我在上面提到的第一点是特征的随机选择，这似乎是田锦镐在Breiman提出RF之前独立提出的(再次参见Wikipedia条目). Ho已经建议仅通过随机特征选择来提高性能.这与您在这里所做的不完全一样(您仍然使用装袋中的引导程序采样方法)，但是您可以通过在RandomForestClassifier()参数中设置bootstrap=False来轻松复制Ho的方法.事实是，根据这项研究，性能差异并不出乎意料...
The RF algorihm is essentially the combination of two independent ideas: bagging, and random selection of features (see the Wikipedia entry for a nice overview). Bagging is essentially my second point above, but applied to an ensemble; random selection of features is my first point above, and it seems that it had been independently proposed by Tin Kam Ho before Breiman's RF (again, see the Wikipedia entry). Ho had already suggested that random feature selection alone improves performance. This is not exactly what you have done here (you still use the bootstrap sampling idea from bagging, too), but you could easily replicate Ho's idea by setting bootstrap=False in your RandomForestClassifier() arguments. The fact is that, given this research, the difference in performance is not unexpected...
要精确地复制中单个树的行为 ，您应同时使用bootstrap=False和max_features=None自变量，即
To replicate exactly the behaviour of a single tree in RandomForestClassifier(), you should use both bootstrap=False and max_features=None arguments, i.e.
clf = RandomForestClassifier(n_estimators=1, max_features=None, bootstrap=False)
在这种情况下，引导采样和随机特征选择均不会发生，并且性能应大致等于单个决策树的性能.
in which case neither bootstrap sampling nor random feature selection will take place, and the performance should be roughly equal to that of a single decision tree.
这篇关于为什么带有一棵树的随机森林比决策树分类器好得多?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！