基于RGB-D深度图像的手势识别研究和人机交互实现

基于RGB-D深度图像的手势识别研究和人机交互实现(任务书,开题报告,论文16000字,代码)
摘要
人工智能已渐渐走进我们的生活，人们对人机交互技术实现的需求也越来越高，而手势识别技术则促进了它的成功。怎样让用户可以轻易方便地经由手势来控制智能机器人是手势识别钻研的大方向。
本文利用了微软公司的Kinect摄像头和优傲机器人公司的UR双臂机器人，设计了一个基于RGBD图像的静态手势识别系统。采用Kinect获取手势图片信息，对手势进行了手型提取与识别，在手型提取这一块采用了深度信息和颜色信息相结合的方法，这样分割出来的手形更加清晰准确；在识别这一块利用了VGG16这一卷积神经网络模型，通过大量的手势图片样本对网络进行训练，最后获得了一个识别率达到90%以上的网络模型。最后用这个训练好的模型进行实时的手势识别，将识别结果传入UR机器人，使得机器人能够根据不同的手势拾取不同的物品。
本文的特色在于没有仅使用深度信息来对手掌进行分割，而是在这基础上又加入了颜色信息，使得手势的提取更加准确，也减少了网络的训练参数。另外在识别阶段采用了卷积神经网络的识别方法，利用神经网络就无需再人工提取手势特征，大大地增加了手势识别的效率。
关键词：手势识别；kinect；深度图像；卷积神经网络
 
Abstract
With artificial intelligence coming into our life, human-computer interaction technology is developing rapidly, and gesture recognition technology used in human-computer interaction process has received widespread attention. How to make it easy and convenient for users to control intelligent robots by gestures is a major goal of gesture recognition research.           
In this paper, a gesture recognition system based on RGBD image is designed by using Microsoft Kinect camera and UR dual-arm robot. Kinect is used to acquire gesture picture information and to extract and recognize gestures. In this part of gesture extraction, depth information and color information are combined to make the segmented hand shape clearer and more accurate. In recognition, VGG16 is used as a convolutional neural network model to train the network through a large number of gesture picture samples, and finally the network is acquired. A network model with recognition rate of more than 90%. Finally, the trained model is used for real-time gesture recognition, and the recognition results are transmitted to the UR robot, so that the robot can pick up different objects according to different gestures.           
The feature of this paper is that not only the depth information is used to segment the palm, but also the color information is added on this basis, which makes the gesture extraction more accurate and reduces the training parameters of the network. In addition, the recognition method of convolution neural network is used in the recognition stage. The use of neural network can greatly increase the efficiency of gesture recognition without manual extraction of gesture features.           
Key words: gesture recognition; kinect; depth image; convolutional neural network
 

目录
摘要    I
Abstract    II
第一章绪论    1
1.1    手势识别的研究背景及意义    1
1.2    手势识别技术的研究概况    2
1.2.1    手势识别的分类    2
1.2.2 手势识别的关键技术    2
1.2.3    手势识别研究的国内外现状    4
1.3    研究内容及目标    4
1.4 论文的组织结构    5
第二章深度图像的获取    6
2.1 Kinect简介    6
2.1.1 Kinect硬件结构    7
2.1.2 Kinect骨骼跟踪    7
2.2 Kinect获取深度图像    8
第三章手势分割    9
3.1 基于肤色检测的手势分割    9
3.2 基于深度信息的手势分割    11
3.2.1 深度距离与实际距离的转换    11
3.2.2 基于阈值分割获取手形    11
3.3 改进后的手势分割方法    12
第四章基于卷积神经网络的静态手势识别    13
4.1 卷积神经网络介绍    13
4.2 VGG16模型    16
4.3 卷积神经网络的训练    17
4.4 卷积神经网络的代码实现    18
第五章手势识别在UR机器人上的应用    21
5.1 系统开发环境    21
5.2 人机交互模块的设计    21
5.2.1 UR5机器人    22
5.2.2 机器人控制模块的设计    23
5.3 人机交互实验结果展示    24
第六章总结与期望    26
参考文献    27
致谢    29