问题描述
ANN(人工神经网络)和 SVM(支持向量机)是监督机器学习和分类的两种流行策略.通常不清楚哪种方法更适合特定项目，我确信答案总是视情况而定".通常，将两者与贝叶斯分类结合使用.
ANN (Artificial Neural Networks) and SVM (Support Vector Machines) are two popular strategies for supervised machine learning and classification. It's not often clear which method is better for a particular project, and I'm certain the answer is always "it depends." Often, a combination of both along with Bayesian classification is used.
关于 ANN 与 SVM 的这些关于 Stackoverflow 的问题已经被问到:
These questions on Stackoverflow have already been asked regarding ANN vs SVM:
ANN之间有什么区别, SVM 和 KNN 在我的分类问题中
在这个问题中，我想具体地了解 ANN(具体来说，多层感知器)的哪些方面可能使其优于 SVM?我问的原因是因为它很容易回答相反的问题:支持向量机通常优于 ANN，因为它们避免了 ANN 的两个主要弱点:
In this question, I'd like to know specifically what aspects of an ANN (specifically, a Multilayer Perceptron) might make it desirable to use over an SVM? The reason I ask is because it's easy to answer the opposite question: Support Vector Machines are often superior to ANNs because they avoid two major weaknesses of ANNs:
(1) 人工神经网络经常收敛于局部最小值而不是全局最小值，这意味着它们有时本质上是错过了大局"(或只见树木不见森林)
(1) ANNs often converge on local minima rather than global minima, meaning that they are essentially "missing the big picture" sometimes (or missing the forest for the trees)
(2) 如果训练时间过长，人工神经网络通常会过拟合，这意味着对于任何给定的模式，人工神经网络可能会开始将噪声视为模式的一部分.
(2) ANNs often overfit if training goes on too long, meaning that for any given pattern, an ANN might start to consider the noise as part of the pattern.
SVM 不会遇到这两个问题中的任何一个.然而，SVM 是否意味着要完全替代 ANN，这一点并不明显.那么，ANN 相对于 SVM 有哪些特定优势，可能使其适用于某些情况?我已经列出了 SVM 相对于 ANN 的特定优势，现在我想查看 ANN 优势列表(如果有的话).
SVMs don't suffer from either of these two problems. However, it's not readily apparent that SVMs are meant to be a total replacement for ANNs. So what specific advantage(s) does an ANN have over an SVM that might make it applicable for certain situations? I've listed specific advantages of an SVM over an ANN, now I'd like to see a list of ANN advantages (if any).
推荐答案
从你提供的例子来看，我假设 ANNs 是指多层前馈网络(简称 FF 网络)，例如多层感知器，因为它们与 SVM 直接竞争.
Judging from the examples you provide, I'm assuming that by ANNs, you mean multilayer feed-forward networks (FF nets for short), such as multilayer perceptrons, because those are in direct competition with SVMs.
这些模型相对于 SVM 的一个具体好处是它们的大小是固定的:它们是参数模型，而 SVM 是非参数的.也就是说，在 ANN 中，你有一堆大小为 h 到 h 取决于特征的数量，加上偏置参数，以及构成模型的那些.相比之下，SVM(至少是核化的)由一组支持向量组成，从训练集中选择，每个支持向量都有一个权重.在最坏的情况下，支持向量的数量正好是训练样本的数量(尽管这主要发生在小训练集或退化情况下)，并且通常其模型大小呈线性比例.在自然语言处理中，支持向量数以万计，每个支持向量具有数十万个特征的 SVM 分类器并非闻所未闻.
One specific benefit that these models have over SVMs is that their size is fixed: they are parametric models, while SVMs are non-parametric. That is, in an ANN you have a bunch of hidden layers with sizes h through h depending on the number of features, plus bias parameters, and those make up your model. By contrast, an SVM (at least a kernelized one) consists of a set of support vectors, selected from the training set, with a weight for each. In the worst case, the number of support vectors is exactly the number of training samples (though that mainly occurs with small training sets or in degenerate cases) and in general its model size scales linearly. In natural language processing, SVM classifiers with tens of thousands of support vectors, each having hundreds of thousands of features, is not unheard of.
此外，与在线 SVM 拟合相比，FF 网络的在线训练非常简单，并且预测可能相当多更快.
Also, online training of FF nets is very simple compared to online SVM fitting, and predicting can be quite a bit faster.
编辑:以上所有内容都与内核化 SVM 的一般情况有关.线性 SVM 是一种特殊情况，因为它们是参数化的，并且允许使用简单的算法(例如随机梯度下降)进行在线学习.
EDIT: all of the above pertains to the general case of kernelized SVMs. Linear SVM are a special case in that they are parametric and allow online learning with simple algorithms such as stochastic gradient descent.
这篇关于人工神经网络与支持向量机相比有哪些优势?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！