[1]王信,汪友生.基于深度学习与传统机器学习的人脸表情识别综述[J].应用科技,2018,45(01):65-72.[doi:10.11991/yykj.201707008]
 WANG Xin,WANG Yousheng.Facial expression recognition based on deep learning and traditional machine learning[J].Applied science and technology,2018,45(01):65-72.[doi:10.11991/yykj.201707008]
点击复制

基于深度学习与传统机器学习的人脸表情识别综述(/HTML)
分享到:

《应用科技》[ISSN:1009-671X/CN:23-1191/U]

卷:
第45卷
期数:
2018年01期
页码:
65-72
栏目:
现代电子技术
出版日期:
2018-02-05

文章信息/Info

Title:
Facial expression recognition based on deep learning and traditional machine learning
作者:
王信 汪友生
北京工业大学 信息学部, 北京 100124
Author(s):
WANG Xin WANG Yousheng
School of Information Science, Beijing University of Technology, Beijing 100124, China
关键词:
人脸表情识别深度学习CNN机器学习计算机视觉图像预处理特征提取特征分类
Keywords:
facial expression recognitiondeep learningCNNmachine learningcomputer visionimage pre-processingfeature extractionfeature classification
分类号:
TP391.41
DOI:
10.11991/yykj.201707008
文献标志码:
A
摘要:
现有的人脸表情识别技术基本局限于传统的机器学习算法,在光照强弱、有遮挡物、姿态变换等情况下,传统的机器学习算法鲁棒性差,难以运用到实际生活中。随着计算机GPU等硬件条件的发展、大数据时代的到来,深度学习在计算机视觉领域备受关注。本文从图像预处理、特征提取、特征分类等方面介绍了传统机器学习算法及其优缺点;从DBN、CNN等主流算法、发展方向、常用开发框架介绍了深度学习算法。最后总结和展望了传统机器学习与深度学习在人脸表情识别上的发展问题与趋势以及后续研究方向。
Abstract:
The existing facial expression recognition technology is limited to the traditional machine learning algorithms basically. The traditional machine learning algorithms have a low robustness in the case of light intensity, obstruction and posture change, which causes some difficulties in the practical application. With the development of the computer hardware conditions such as GPU and the development of the big data, deep learning became the focus in the field of computer vision. This paper introduces the traditional machine learning algorithm, its merits and shortcomings from such aspects as image preprocessing, feature extraction and feature classification; describes the deep learning algorithm from such aspects as the mainstream algorithms including DBN and CNN, development direction and common development framework; finally, summarizes and forecasts the development and trends of traditional machine learning and deep learning in facial expression recognition as well as the follow-up research direction.

参考文献/References:

[1] MEHRABIAN A, RUSSELL J A. An approach to environmental psychology[M]. Cambridge, MA:MIT, 1974.
[2] SUWA M, SUGIE N, FUJIMORA K. A preliminary note on pattern recognition of human emotional expression[C]//Proceedings of the 4thInternational Joint Conference on Pattern Recognition. Kyoto, Japan, 1978:408-410.
[3] EKMAN P, FRIESEN W V, HAGERJ C. Facial action coding system(FACS)[M]. Manual, Salt Lake City (USA):A Human Face, 2002.
[4] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe, Nevada,2012:1097-1105.
[5] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Delving deep into rectifiers:surpassing human-level performance on ImageNetclassification[C]//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago, Chile, 2015:1026-1034.
[6] LYONSM J, KAMACHIM, GYOBA J. The Japanese Female FacialExpression(JAFFE)database[EB/OL]. http://www.kasrl.org/jaffe.html.
[7] 薛雨丽, 毛峡, 张帆. BHU人脸表情数据库的设计与实现[J]. 北京航空航天大学学报, 2007, 33(2):224-228.
[8] LUCEY P, COHN J F, KANADE T, et al. The Extended Cohn-Kanade Dataset (CK+):a complete dataset for action unit and emotion-specified expression[C]//Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Recognition-Workshops. San Francisco, CA, USA, 2010:94-101.
[9] VALSTAR M F, PANTICM. Induced disgust, happiness and surprise:an addition to the MMI facialexpression database[C]//Proceedings of Int’l Conference on Language Resources and Evaluation, Workshop on EMOTION. Malta, 2010:65-70.
[10] YIN Lijun, WEI Xiaozhou, SUN Yi, et al. A 3D facial expression database for facial behavior research[C]//Proceedings of the 7th International Conference on Automatic Face and Gesture Recognition. Southampton, UK, 2006:211-216.
[11] VALSTAR M F, MEHU M, JIANG Bihan, et al. Meta-analysis of the first facial expression recognition challenge[J]. IEEE transactions on systems, man, part B (cybernetics), 2012, 42(4):966-979.
[12] GOODFELLOW I J, ERHAN D, CARRIER P L, et al. Challenges in representation learning:a report on three machine learning contests[M]//LEE M, HIROSEA,HOU Zengguan, et al. Neural Information Processing. Berlin Heidelberg:Springer, 2013.
[13] YANGM H, KRIEGMAN D J, AHUJA N. Detecting faces inimages:a survey[J]. IEEE transactions on patternanalysis and machine intelligence, 2002, 24(1):34-58.
[14] LEE T S. Image representation using 2D Gabor wavelets[J]. IEEE transactions on pattern analysis and machine intelligence, 1996, 18(10):959-971.
[15] YU Jiangang, BHANU B. Evolutionary feature synthesis for facial expression recognition[J]. Pattern recognition letters, 2006, 27(11):1289-1298.
[16] LIU Shuaishi, TIAN Yantao, WAN Chuan. Facial expression recognition method based on Gabor multi-orientationfeatures fusion and block histogram[J]. Actaautomaticasinica, 2011, 37(12):1455-1463.
[17] WEN Zhen, HUANG T S. Capturing subtle facial motions in 3D face tracking[C]//Proceedings of the 9th IEEE International Conference on Computer Vision. Nice, France, 2003:1343-1350.
[18] AHONEN T, HADID A, PIETIKÄINEN M. Face recognition with local binary patterns[M]//PAJDLA T, MATASJ. Computer Vision-ECCV 2004. BerlinHeidelberg:Springer, 2004:469-481.
[19] GUO Zhenhua, ZHANG Lei, ZHANG D. A completed modeling of local binary pattern operator for texture classification[J]. IEEE transactions on image processing, 2010, 19(6):1657-1663.
[20] GUO Yimo, ZHAO Guoying, PIETIKÄINEN M. Discriminative features for texture description[J]. Pattern recognition, 2012, 45(10):3834-3843.
[21] FENG X, PIETIKÄINEN M, HADID A. Facial expression recognition based on local binary patterns[J]. Pattern recognition and image analysis, 2007, 17(4):592-598.
[22] 周宇旋, 吴秦, 梁久祯, 等. 判别性完全局部二值模式人脸表情识别[J]. 计算机工程与应用, 2017, 53(4):163-169,194.
[23] JABID T, KABIR M H, CHAE O. Robust facial expression recognition based on local directional pattern[J]. ETRI journal, 2010, 32(5):784-794.
[24] LONARE A, JAIN S V.A survey on facial expression analysis for emotion recognition[J]. International journal of advanced research in computer and communication engineering, 2013, 2(12):4647-4650
[25] COOTES T F, TAYLOR C J, COOPER D H, et al. Active shape models-their training and application[J]. Computer vision and image understanding, 1995, 61(1):38-59.
[26] 黄建, 李文书, 高玉娟. 人脸表情识别研究进展[J]. 计算机科学, 2016, 43(S2):123-126.
[27] CRISTINACCE D, COOTES T, SCOTT I. A multi-stage approach to facial featuredetection[C]//Proceedings of the British Machine Vision Conference. Kingston,2004:231-240.
[28] 侯婕. 人脸表情计算技术研究[D]. 苏州:苏州大学, 2014.
[29] SAATCI Y, TOWN C. Cascaded classification of gender and facial expression using active appearance models[C]//Proceedings of the 7th International Conference on Automatic Face and Gesture Recognition. Southampton, UK, 2006:393-398.
[30] XU Chao, DONG CAICHAO, FENG ZHIYONG, et al. Facial expression pervasive analysis based on Haar-like features and SVM[M]//KHACHIDZEV,WANG T, SIDDIQUI S, et al. Contemporary Research on E-business Technology and Strategy. BerlinHeidelberg:Springer, 2012:521-529.
[31] 周书仁. 人脸表情识别算法分析与研究[D]. 长沙:中南大学, 2009.
[32] LIEW C F, YAIRI T. Facial expression recognition and analysis:a comparison study of feature descriptors[J]. IPSJ transactions on computer vision and applications, 2015, 7:104-120.
[33] 姜波, 解仑, 刘欣, 等. 光流模值估计的微表情捕捉[J]. 浙江大学学报:工学版, 2017, 51(3):577-583, 589.
[34] 赵晓健, 曾晓勤. 基于稠密光流轨迹和稀疏编码算法的行为识别方法[J]. 计算机应用, 2016, 36(1):181-187.
[35] TIE Yun, GUAN Ling. A deformable 3-D facial expression model for dynamic human emotional state recognition[J]. IEEE transactions on circuits and systems for video technology, 2013, 23(1):142-157.
[36] 刘玉, 王敬东, 李鹏. 一种基于SIFT和KLT相结合的特征点跟踪方法研究[J]. 宇航学报, 2011, 32(7):1618-1625.
[37] SALAKANIDOU F, MALASSIOTIS S. Real-time 2D+3D facial action and expression recognition[J]. Pattern recognition, 2010, 43(5):1763-1775.
[38] HE Jun, CAI Jianfeng, FANG Lingzhi, et al. Facial expression recognition based on LBP/VAR and DBN model[J]. Application research of computers, 2016, 33(8):2509-2513.
[39] ZHAN Yongzhao, CHENG Keyang, CHEN Yabi, et al. A new classifier for facial expression recognition:fuzzy buried Markov model[J]. Journal of computer science and technology, 2010, 25(3):641-650.
[40] 李栋. 基于局部纹理特征融合的面部表情识别方法研究[D]. 长春:吉林大学, 2014.
[41] 李文书, 何芳芳, 钱沄涛, 等. 基于Adaboost-高斯过程分类的人脸表情识别[J]. 浙江大学学报:工学版, 2012, 46(1):79-83.
[42] 解仑, 卢亚楠, 姜波, 等. 基于人脸运动单元及表情关系模型的自动表情识别[J]. 北京理工大学学报, 2016, 36(2):163-169.
[43] DOMINGOS P. A few useful things to know about machine learning[J]. Communications of the ACM, 2012, 55(10):78-87.
[44] 刘栋, 李素, 曹志冬. 深度学习及其在图像物体分类与检测中的应用综述[J]. 计算机科学, 2016, 43(12):13-23.
[45] BASTIEN F, LAMBLIN P, PASCANU R, et al. Theano:new features and speed improvements[J]. arXiv:1211.5590, 2012.
[46] ABADI M, BARHAM P, CHEN Jianmin, et al. TensorFlow:a system for large-scale machine learning[C]//Proceedings of the 12th USENIX conference on Operating Systems Design and Implementation.Savannah, GA, USA,2016.
[47] SUI Xudong, ZHANG Jinfang, HU Xiaohui, et al. Monitoring target through satellite images by using deep convolutional networks[M]//LEE R. Software Engineering Research, Management and Applications. Cham:Springer, 2016.
[48] JIAYangqing, SHELHAMER E, DONAHUE J, et al. Caffe:convolutional architecture for fast feature embedding[J]. Arxiv:1408.5093, 2014.
[49] BOJARSKI M, DEL TESTA D, DWORAKOWSKI D, et al. End to end learning for self-driving cars[J]. arXiv:1604.07316,2016.
[50] CS231nconvolutionalneuralnetworksforvisualrecognition[EB/OL].http://cs231n.github.io/neural-networks-2/.
[51] ZHAO Shichao, LIU Yanbin, HAN Yahong, et al. Pooling the convolutional layers in deep ConvNets for video action recognition[J]. arXiv:1511.02126, 2015.
[52] 薛昆南. 基于卷积神经网络的视觉识别研究[D]. 广州:华南农业大学, 2016.
[53] LAJEVARDI S M, HUSSAIN Z M. Higher order orthogonal moments for invariant facial expression recognition[J]. Digital signal processing, 2010, 20(6):1771-1779.
[54] YU Zhiding, ZHANG Cha. Image based static facial expression recognition with multiple deep network learning[C]//Proceedings of 2015 ACM on International Conference on Multimodal Interaction. Seattle, Washington, USA, 2015:435-442.
[55] LOPES A T, DE AGUIAR E, OLIVEIRA-SANTOS T. A facial expression recognition system using convolutional networks[C]//Proceedings of the 28th SIBGRAPI Conference on Graphics, Patterns and Images. Salvador, Brazil, 2015:273-280.
[56] WANG Jieru, YUAN Chun. Facial expression recognition with multi-scale convolution neural network[M]//CHEN Enqing, GONG Yihong, TIE Yun. Advances in Multimedia Information Processing-PCM 2016. Cham:Springer, 2016.
[57] ZHAO Xiaoming, SHI Xugan, ZHANG Shiqing. Facial expression recognition via deep learning[J]. IETE technical review, 2015, 32(5):347-355.
[58] LI Chen, WEI Wei, WANG Jingzhong, et al. Face recognition based on deep belief network combined with center-symmetric local binary pattern[M]//PARK J J, JIN Hai, JEONG Y S, et al. Advanced Multimedia and Ubiquitous Engineering. Singapore:Springer, 2016.本文引用格式:王信,汪友生. 基于深度学习与传统机器学习的人脸表情识别综述[J]. 应用科技, 2018, 45(1):65-72.WANG Xin, WANG Yousheng. Facial expression recognition based on deep learning and traditional machine learning[J]. Applied science and technology, 2018, 45(1):65-72.

相似文献/References:

[1]宋佳蓉,杨忠,张天翼,等.基于卷积神经网络和多类SVM的交通标志识别[J].应用科技,2018,45(05):71.[doi:10.11991/yykj.201710009]
 SONG Jiarong,YANG Zhong,ZHANG Tianyi,et al.Traffic signs recogniton based on convolutional neural networks and multi-class SVM[J].Applied science and technology,2018,45(01):71.[doi:10.11991/yykj.201710009]
[2]王立新,江加和.基于深度学习的显著性区域的图像检索研究[J].应用科技,2018,45(06):63.[doi:10.11991/yykj.201803012]
 WANG Lixin,JIANG Jiahe.Research on image retrieval of saliency region based on deep learning[J].Applied science and technology,2018,45(01):63.[doi:10.11991/yykj.201803012]

备注/Memo

备注/Memo:
收稿日期:2017-07-27。
基金项目:中国博士后科学基金项目(2017M610731).
作者简介:王信(1992-),男,硕士研究生;汪友生(1966-),男,副教授,博士.
通讯作者:汪友生,E-mail:wangyousheng@bjut.edu.cn
更新日期/Last Update: 2018-03-14