问题描述
我目前正在开发一个强大的手部检测系统.
I am currently working on a system for robust hand detection.
第一步是拍一张手的照片(在 HSV 颜色空间中)，手放在一个小矩形中，以确定皮肤颜色.然后我应用阈值过滤器将所有非皮肤像素设置为黑色，将所有皮肤像素设置为白色.
The first step is to take a photo of the hand (in HSV color space) with the hand placed in a small rectangle to determine the skin color. I then apply a thresholding filter to set all non-skin pixels to black and all skin pixels white.
到目前为止，它工作得很好，但我想问是否有更好的方法来解决这个问题?例如，我发现有几篇论文提到了白种人的具体色彩空间，但没有一篇文章与亚洲/非洲/白种人的色调进行比较.
So far it works quite well, but I wanted to ask if there is a better way to solve this? For example, I found a few papers mentioning concrete color spaces for caucasian people, but none with a comparison for asian/african/caucasian color-tones.
顺便说一句，我正在通过 Python 绑定使用 OpenCV.
By the way, I'm working with OpenCV via Python bindings.
推荐答案
你看过 Gary Bradski 的 camshift 论文了吗?您可以从 这里
Have you taken a look at the camshift paper by Gary Bradski? You can download it from here
我在一年前使用皮肤检测算法来检测皮肤区域以进行手部跟踪，它非常强大.这取决于你如何使用它.
I used the the skin detection algorithm a year ago for detecting skin regions for hand tracking and it is robust. It depends on how you use it.
使用颜色进行跟踪的第一个问题是，当人们具有不同的肤色时，它对光照变化或如您提到的那样不可靠.然而，正如论文中提到的那样，这可以通过以下方式轻松解决:
The first problem with using color for tracking is that it is not robust to lighting variations or like you mentioned, when people have different skin tones. However this can be solved easily as mentioned in the paper by:
- 将图像转换为 HSV 颜色空间.
- 抛开 V 通道，考虑 H 和 S 通道，因此照明变化的折扣.
- 阈值像素由于其不稳定性而具有低饱和度.
- 将选定的皮肤区域分箱成二维直方图.(OpenCV"s calcHist函数)此直方图现在充当皮肤模型.
- 计算反投影"(即使用直方图计算概率"图像中的每个像素都有肤色)使用 calcBackProject.皮肤地区将具有较高的价值.
- 然后您可以使用 meanShift 寻找 2D 的模式概率"图由 backproject 生成或检测 blob高概率".
- Convert image to HSV color space.
- Throw away the V channel and consider the H and S channel and hencediscount for lighting variations.
- Threshold pixels with low saturation due to their instability.
- Bin the selected skin region into a 2D histogram. (OpenCV"s calcHistfunction) This histogram now acts as a model for skin.
- Compute the "backprojection" (i.e. use the histogram to compute the "probability"that each pixel in your image has the color of skin tone) using calcBackProject. Skinregions will have high values.
- You can then either use meanShift to look for the mode of the 2D"probability" map generated by backproject or to detect blobs ofhigh "probability".
在 HSV 中抛弃 V 通道并仅考虑 H 和 S 通道确实足以(令人惊讶地)检测不同的肤色和不同的光照变化.一个优点是它的计算速度很快.
Throwing away the V channel in HSV and only considering H and S channels is really enough (surprisingly) to detect different skin tones and under different lighting variations. A plus side is that its computation is fast.
这些步骤和对应的代码可以在原文中找到OpenCV 书.
These steps and the corresponding code can be found in the original OpenCV book.
顺便说一句，我之前也使用过高斯混合模型 (GMM).如果您只考虑颜色，那么我会说使用直方图或 GMM 并没有太大区别.事实上，直方图会表现得更好(如果你的 GMM 不是为了考虑照明变化等而构建的).如果您的样本向量更复杂(即您考虑其他特征)，则 GMM 很好，但速度方面的直方图要快得多，因为使用直方图计算概率图本质上是一个表格查找，而 GMM 需要执行矩阵计算(对于维度 >1 在多维高斯分布的公式中)，这对于实时应用来说可能很耗时.
As a side note, I've also used Gaussian Mixture Models (GMM) before. If you are only considering color then I would say using histograms or GMM makes not much difference. In fact the histogram would perform better (if your GMM is not constructed to account for lighting variations etc.). GMM is good if your sample vectors are more sophisticated (i.e. you consider other features) but speed-wise histogram is much faster because computing the probability map using histogram is essentially a table lookup whereas GMM requires performing a matrix computation (for vector with dimension > 1 in the formula for multi-dimension gaussian distribution) which can be time consuming for real time applications.
因此，总而言之，如果您只是尝试使用颜色检测皮肤区域，那么请使用直方图方法.您也可以调整它以考虑局部梯度(即梯度直方图，但可能不会达到 Dalal 和 Trigg 的人体检测算法的全部范围.)以便它可以区分皮肤和具有相似颜色的区域(例如纸板或木制家具) 使用局部纹理信息.但这需要更多的努力.
So in conclusion, if you are only trying to detect skin regions using color, then go with the histogram method. You can adapt it to consider local gradient as well (i.e. histogram of gradients but possibly not going to the full extent of Dalal and Trigg's human detection algo.) so that it can differentiate between skin and regions with similar color (e.g. cardboard or wooden furniture) using the local texture information. But that would require more effort.
有关如何使用直方图进行皮肤检测的示例源代码，您可以查看 OpenCV 的页面 这里.但请注意，该网页上提到他们只使用色调通道，并且同时使用色调和饱和度会产生更好的效果.
For sample source code on how to use histogram for skin detection, you can take a look at OpenCV"s page here. But do note that it is mentioned on that webpage that they only use the hue channel and that using both hue and saturation would give better result.
要了解更复杂的方法，您可以查看 Margaret Fleck 和 David Forsyth 的检测裸人"的工作.这是检测同时考虑颜色和纹理的皮肤区域的早期工作之一.详细信息可以在这里找到.
For a more sophisticated approach, you can take a look at the work on "Detecting naked people" by Margaret Fleck and David Forsyth. This was one of the earlier work on detecting skin regions that considers both color and texture. The details can be found here.
可以找到一个很好的与计算机视觉和图像处理相关的源代码资源，其中恰好包含用于视觉跟踪的代码这里.不是，它不是 OpenCV.
A great resource for source code related to computer vision and image processing, which happens to include code for visual tracking can be found here. And not, its not OpenCV.
希望这会有所帮助.
这篇关于通过计算机视觉进行稳健的手部检测的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！