星火1.5.1，MLLib随机森林概率

问题描述

我使用的Spark 1.5.1与MLLib。我建立使用MLLib随机林模型，现在使用的模式去做prediction。我能找到使用。predict功能predict类（0.0或1.0）。但是，我找不到检索的概率（见附件截屏）的功能。我以为火花1.5.1随机森林将提供的可能性，我在这里缺少什么？

I am using Spark 1.5.1 with MLLib. I built a random forest model using MLLib, now use the model to do prediction. I can find the predict category (0.0 or 1.0) using the .predict function. However, I can't find the function to retrieve the probability (see the attached screenshot). I thought spark 1.5.1 random forest would provide the probability, am I missing anything here?

推荐答案

不幸的是此功能不可用MLlib，在另一方面，你可以用ML的RandomForestClassifier找到它：

Unfortunately this feature is not available with MLlib, on other hand you can find it with ML's RandomForestClassifier :

import org.apache.spark.ml.Pipeline
import org.apache.spark.ml.classification.RandomForestClassifier
import org.apache.spark.ml.feature.{IndexToString, StringIndexer, VectorIndexer}
import org.apache.spark.mllib.util.MLUtils

// Load and parse the data file, converting it to a DataFrame.
val data = MLUtils.loadLibSVMFile(sc, "data/mllib/sample_libsvm_data.txt").toDF()

// Index labels, adding metadata to the label column.
// Fit on whole dataset to include all labels in index.
val labelIndexer = new StringIndexer().setInputCol("label").setOutputCol("indexedLabel").fit(data)

// Automatically identify categorical features, and index them.
// Set maxCategories so features with > 4 distinct values are treated as continuous.
val featureIndexer = new VectorIndexer().setInputCol("features").setOutputCol("indexedFeatures").setMaxCategories(4).fit(data)

// Split the data into training and test sets (30% held out for testing)
val Array(trainingData, testData) = data.randomSplit(Array(0.7, 0.3))

// Train a RandomForest model.
val rf = new RandomForestClassifier().setLabelCol("indexedLabel").setFeaturesCol("indexedFeatures").setNumTrees(10)

// Convert indexed labels back to original labels.
val labelConverter = new IndexToString().setInputCol("prediction").setOutputCol("predictedLabel").setLabels(labelIndexer.labels)

// Chain indexers and forest in a Pipeline
val pipeline = new Pipeline().setStages(Array(labelIndexer, featureIndexer, rf, labelConverter))

// Train model.  This also runs the indexers.
val model = pipeline.fit(trainingData)

// Make predictions.
val predictions: DataFrame = model.transform(testData)

predictions.printSchema

您会注意到的概率列是present：

You'll notice that the probability column is present :

root
 |-- label: double (nullable = false)
 |-- features: vector (nullable = true)
 |-- indexedLabel: double (nullable = true)
 |-- indexedFeatures: vector (nullable = true)
 |-- rawPrediction: vector (nullable = true)
 |-- probability: vector (nullable = true)
 |-- prediction: double (nullable = true)
 |-- predictedLabel: string (nullable = true)

注意：这个例子是从的。

这是一些输出列一些解释：

And here is some explanation on some output columns :

• predictionCol 重新presents的predicted标签。


• 原材料predictionCol 重新presents长度＃类的载体，以训练实例标签在树节点这使得$ P $计数pdiction（仅适用于分类只）。


• probabilityCol 重新presents等于原材料prediction 标准化长度＃类的概率向量以多项分布（可带分类只）。

• predictionCol represents the predicted label .
• rawPredictionCol represents a Vector of length # classes, with the counts of training instance labels at the tree node which makes the prediction (available for Classification only).
• probabilityCol represents the probability Vector of length # classes equal to rawPrediction normalized to a multinomial distribution (available with Classification only).

这篇关于星火1.5.1，MLLib随机森林概率的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！