- 无标题文档
查看论文信息

中文题名:

 

基于动作识别的人机协作任务规划方法研究

    

姓名:

 李吉轩    

学号:

 1049721904035    

保密级别:

 公开    

论文语种:

 chi    

学科代码:

 080200    

学科名称:

 工学 - 机械工程    

学生类型:

 硕士    

学校:

 武汉理工大学    

院系:

 交通与物流工程学院    

专业:

 机械工程    

研究方向:

 人机协作    

第一导师姓名:

 赵章焰    

第一导师院系:

 交通与物流工程学院    

完成日期:

 2022-03-23    

答辩日期:

 2022-05-13    

中文关键词:

 

人机协作 ; 动作识别 ; 任务规划 ; 人体疲劳度 ; 人工蜂群算法

    

中文摘要:

人机协作制造是指人和机器人共享同一个工作空间并完成制造任务的过程,其结合了机器人与人的优点,提高了生产效率。现代制造生产作业具有多品种、小批量的特点。而人机协作既能利用人类的灵活性和决策能力又能发挥机器人抗疲劳、高负荷、作业效率持续稳定的优点。在人机协作场景中,人类在长时间的重复工作下会造成疲劳度的积累并影响协作效率,在传统的任务编排过程中无法充分考虑人因工程要素并动态改变任务编排。现阶段人机协作任务规划方法上存在人因要素不足、系统动态性不足、任务规划实时性不强等缺陷。因此,本文研究人机协作任务规划并动态调整装配任务节律以稳定人机装配节律具有重要意义。针对上述问题,本文重点研究的人机协作装配任务动态规划问题,其主要研究工作如下:

(1)构建了基于人员疲劳度的人机协作装配模型。本文建立了面向人机协作场景中装配过程的零件装配信息模型与人体疲劳度模型,在此基础上结合人类与机器人在协作装配场景下的不同优势,提出了基于人员疲劳度的人机协作装配模型。

(2)在人机协作装配模型基础上,实现了在协作场景中对人体疲劳度的动态监测与工序状态的实时识别。应用了神经网络算法Mediapipe与YOLO v4算法构建了面向人机协作任务的动态评价模型。在该模型中采用基于Mediapipe的人体疲劳度检测算法,实现了对人体疲劳度识别;并通过粒子群优化支持向量机PSO-SVM算法与手部21个位点的三维坐标信息实现了对装配过程中的人体手部动作识别;提出了在协作装配场景下的基于YOLO v4算法的装配工具识别算法,结合手部动作识别结果完成了对装配工序的检测;实验结果表明该模型的综合识别速度达到24FPS,综合识别成功率达到91.28%。

(3)提出了面向人机协作任务的动态评价模型,针对该模型归纳为 “成本-时间-难度”的三维度多目标优化问题。本文在原始版本的人工蜂群的基础上,在可行解编码解码、种群初始化策略、精英蜂群生成策略、跟随蜂生成策略与局部搜索策略等方面进行改进,其与另外三类常用算法对比具有更好的收敛性能与求解稳定性,满足人机协作场景中的人机任务规划需求。

(4)搭建了面向人机协作装配的人机协作系统。利用协作机器人、摄像头等硬件设备,设计了一款协作装配场景下的面向人机任务规划的人机协作系统。

实验结果表明,本文所提出的算法通过更合理的任务编排方式减缓了装配作业对人体疲劳积累效应,将重复疲劳转化为间歇疲劳,提升了23%装配绩效。

参考文献:

[1] 中华人民共和国国家统计局中国统计年鉴2020[M/OL].[2020]. http://www.stats.gov.cn/tjsj/ndsj/2020/indexch.htm.

[2] 刘检华, 孙清超, 程晖, 等. 产品装配技术的研究现状、技术内涵及发展趋势[J].机械工程学报, 2018, 54(11): 2-28.

[3] 夷萍.华为、海尔、GE、西门子——互联网+转型,学这四家就够了[J]. 商学院, 2015,2015(08): 95-96.

[4] 周文博. 智能协作机器人与柔性产线技术[J]. 电子技术与软件工程,2017, 2017(20):94.

[5] Villani V, Pini F, Leali F, et al. Survey on human–robot collaboration in industrial settings: Safety, intuitive interfaces and applications[J]. Mechatronics, 2018, 55: 248-266.

[6] Panagiota Tsarouchi, George Michalos, Sotiris Makris, Thanasis Athanasatos, Konstantinos Dimoulas & George Chryssolouris. On a human–robot workplace design and task allocation system[J]. International Journal of Computer Integrated Manufacturing, 2017: 1-8.

[7] Li Z, Moran P, Dong Q, et al. Development of a telenursing mobile manipulator for remotecare-giving in quarantine areas[C]// Proceedings of IEEE International Conference on Robotics and Automation. 2017: 3581-3586.

[8] SA Seshia, D Sadigh, SS Sastry. Formal Methods for Semi-Autonomous Driving[C]// Proceedings of Design Automation Conference. IEEE, 2015: 148.

[9] Faber M, Bützler J, Schlick C M. Human-robot cooperation in future production systems:Analysis ofrequirements for designing an ergonomic work system[J]. Proceding Manufactoring, 2015, 3: 510-517.

[10] Villani V, Sabattini L, Czerniaki J N, et al. Towards modern inclusive factories: A methodology for the development of smart adaptive human-machine interfaces[A]// 2017 22nd IEEE International Conference on Emerging Technologies and Factory Automation (ETFA)[C]. 2017: 1-7.

[11] Johannsmeier L, Haddadin S. A hierarchical human-robot interaction-planning framework fortask allocation in collaborative industrial assembly processes[J]. IEEE Robotics and Automa-tion Letters, 2016, 2(1): 41-48.

[12] S El Zaatari, M Marei, W Li, Z Usman Cobot Programming for Collaborative Industrial Tasks:An Overview [J]. Robotics and Autonomous Systems, 2019,116:162-180.

[13] 贾计东, 张明路. 人机安全交互技术研究进展及发展趋势[J]. 机械工程学报, 2020, 56(03): 16-30.

[14] Lambrechtj, Kruger J. Spatial programm ing for indu strial robots based ongestures and Augmented Reality[C]. 2012 IEEE/RSJ Inter national Conferenceon Intelligent Robots and Systems. 2012: 466 472.

[15] STOLT A, CARLSON f B, ARDAKANI MMG, et al. Sensorless friction-compensated passive lead-through programming for industrial robots[A]// 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)[C]. 2015: 3530-3537.

[16] I El Makrini, SA Elprama.Working with walt: How a cobot was developed and inserted on an auto assembly line[J]. IEEE Robotics & Automation Magazine. 2018, PP(99):1-1.

[17] 刘洋, 孙恺. 协作机器人的研究现状与与技术发展分析[J]. 北方工业大学学报, 2017, 29(02): 76-85.

[18] 吴丹, 赵安安, 陈恳,等. 协作机器人及其在航空制造中的应用综述[J]. 航空制造技术, 2019, 62(10): 24-34.

[19] 蒋璐. 人机协作中的运动规划与移动机械臂自适应控制[D]. 合肥:中国科学技术大学, 2021.

[20] 焦龙. 基于意图识别的人机协作控制研究[D]. 哈尔滨: 哈尔滨理工大学, 2021.

[21] 彭涛. 基于sEMG的人机协作机械臂变阻抗示教技术研究[D]. 哈尔滨: 哈尔滨工业大学, 2018.

[22] Morato C, Kaipa K N, Zhao B, et al. Toward Safe Human Robot Collaboration by Using Multiple Kinects Based Real-Time Human Tracking[J]. Journal of Computing and Information Science in Engineering, 2014, 14(1).

[23] Nikolaos Nikolakis,Vasilis Maratos,Sotiris Makris. A cyber physical system (CPS) approach for safe human-robot collaboration in a shared workplace[J]. Robotics and Computer Integrated Manufacturing. 2019,56:233-243.

[24] 段峰, 王耀南, 雷晓峰, 吴立钊, 谭文. 机器视觉技术及其应用综述[J]. 自动化博览, 2002(03): 62-64.

[25] 宋春华, 彭泫知. 机器视觉研究与发展综述[J]. 装备制造技术, 2019(06): 213-216.

[26] A M Husein1, Calvin1, David Halim1, et al. Motion detect application with frame difference method on a surveillance camera[J]. Journal of Physics: Conference Series, 2019,1230:12-17.

[27] Elisheva Bonchek-Dokowa, Gal A. Kaminka.Towards Computational Models of Intention Detection and Intention Prediction[J]. Cognitive Systems Research, 2014,28:44-79.

[28] 邓利群. 三维人体动作识别及其在交互舞蹈系统上的应用[D]. 合肥: 中国科学技术大学, 2012.

[29] 涂武强. 基于人体动作识别的人机协作安全策略研究[D]. 哈尔滨: 哈尔滨工业大学, 2020.

[30] Coupete E, Moutarde F, Manitsaris S, et al. Gesture Recognition Using a Depth Camera for Human Robot Collaboration on Assembly Line[J]. Procedia Manufacturing, 2015: 518-525.

[31] Ruan X, Tian C. Dynamic gesture recognition based on improved DTW algorithm[C]. Proceedings of IEEE International Conference on Mechatronics and Automation. New York:IEEE, 2015.

[32] Yan S, Xiong Y, Lin D. Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition[J]. arXiv: 1801.07455 [cs], 2018.

[33] Camillo Lugaresi, Jiuqiang Tang, Hadon Nash, et al. Mediapipe: A Framework for Building Perception Pipelines[J]. arXiv:1906.08172[cs], 2019.

[34] Fan Zhang, Valentin Bazarevsky, Andrey Vakunov,et al.Mediapipe Hands: On-device Real-time Hand Tracking,[J]. arXiv:2006.10214[cs], 2019.

[35] 姜杰文. 基于手势识别的协作机器人人机交互系统设计[D]. 大连: 大连理工大学, 2019.

[36] 寇大磊, 权冀川, 张仲伟. 基于深度学习的目标检测框架进展研究[J]. 计算机工程与应用, 2019, 55(11): 25-34.

[37] 鞠默然, 罗海波, 王仲博, 等. 改进的YOLO V3算法及其在小目标检测中的应用[J]. 光学学报, 2019, 39(07): 253-260.

[38] 赵德安, 吴任迪, 刘晓洋, 等. 基于YOLO深度卷积神经网络的复杂背景下机器人采摘苹果定位[J]. 农业工程学报, 2019, 35(03): 164-173.

[39] 高宗, 李少波, 陈济楠, 等. 基于YOLO网络的行人检测方法[J]. 计算机工程, 2018, 44(05): 215-219.

[40] Alexey Bochkovskiy, Chien-Yao Wang, Hong-Yuan Mark Liao.YOLOv4: Optimal Speed and Accuracy of Object Detection[J]. arXiv: 2004.10934 [cs], 2020.

[41] 刘胧, 郭钢, 高敏. FMS人、机功能分配决策支持系统的研究[C]// 第三届全国人—机—环境系统工程学术会议论文集. 贵州:中国系统工程学会,1997:280-284.

[42] Jordan, Nehemiah.Allocation of functions between man and machines in automated systems[J]. Journal of Applied Psychology, 1962, 47(3): 161–165.

[43] 刘晨, 完颜笑如, 庄达民. 基于大型客机操作程序的人机功能分配研究[J]. 民用飞机设计与研究, 2013(03): 31-35+79.

[44] Arai T, Kato R, Fujita M. Assessment of Operator Stress Induced by Robot Collaboration in Assembly[J]. CIRP Annals - Manufacturing Technology (S0007-8506), 2010, 59(1): 5-8.

[45] 徐红旗, 史冀鹏, 张欣, 等. 美国NIOSH手工提举方程在中国的验证研究[J]. 人类工效学, 2018,(3): 31-36.

[46] 李凯. 基于人员疲劳度的人机协作拆卸序列规划研究[D]. 武汉: 武汉理工大学, 2020.

[47] 王恺. 人与机器人协作中的肌肉疲劳预测理论与实验研究[D]. 广州: 华南理工大学, 2019.

[48] Paas F,Van Merriënboer J J G.Variability of Worked Examples and Transfer of Geometrical Problem-solving Skills:A Cognitive-load Approach[J]. Journal of Educational Psychology, 1994, 86(1): 122-133.

[49] Neuhöfer J A,Kausch B,Schlick C M.Embedded Augmented Reality Training System for Dynamic Human-Robot Cooperation[J]. Human Dimensions in Embedded Virtual Simulation,2009, 13:20-22.

[50] 王强, 范秀敏, 何其昌, 等. 面向人机协作装配规划的AR仿真交互与人因评估[J]. 系统仿真学报, 2021, 33(02): 389-400.

[51] 孔繁森, 高天宇, 李惠敏, 等. 考虑任务复杂性的人机联合任务分配问题研究[J]. 机械工程学报, 2021, 57(07): 204-214.

[52] 高天宇. 基于复杂性的人机协作装配任务分配问题研究[D]. 吉林: 吉林大学, 2020.

[53] 南函池. 考虑工作胜任度的人机协同任务分配模型与人机协作策略研究[D]. 杭州: 杭州电子科技大学, 2020.

[54] Colgate J E,Edward J,Peshkin M A,et al. Cobots:Robots for collaboration with human operators[A]// 1996 The International Mechanical Engineering Congress and Exhibition[C].58: 433-439.

[55] Heydaryan S , Bedolla J S, Belingardi G. Safety design and development of a human-robot collaboration assembly process in the automotive industry[J]. Applied Sciences, 2018, 8(3): 344–365.

[56] Schoberger D, Lindorferr, Froschauer R. Modeling workflows for industrial robots considering human-robot collaboration[C]// Proceedings of 2018 IEEE 16th International Conference on Industrial Informatics. New York: IEEE, 2018.

[57] 于嘉鹏, 邢宇飞, 王成恩. 基于扩展干涉矩阵的几何可拆卸性判别方法[J]. 机械工程学报, 2011, 47(21): 146-156.

[58] Kwon W Y, Suh I H. Planning of Proactive Behaviors for Human-robot Cooperative Tasks under Uncertainty[J]. Knowledge-Based Systems, 2014,72:81-95.

[59] Sekiyama K Huang J, et al. An Assembly Strategy Scheduling Method for Human and Robot Coordinated Cell Manufacturing[J]. Intermational Journal of Intelligent Computing & Cybemnetics, 2011, 4(4):487-510.

[60] 王杰. 面向人机协作装配过程的任务分配优化[D]. 武汉: 华中科技大学, 2018.

[61] 张里博. 面向人机协作机制的三支决策模型及其应用研究[D]. 南京: 南京大学, 2019.

[62] 刘久兵. 三支直觉模糊决策方法及在人机任务分配中的应用研究[D]. 南京: 南京大学, 2019.

[63] Chacón A,Ponsa P,Angulo,et al. Cognitive Interaction Analysis in Human–Robot Collaboration Using an Assembly Task[J]. Electronics, 2021, 10:13-17.

[64] Dawson D, Noy Y I, Härmä M, et al. Modelling fatigue and the use of fatigue models in work settings[J]. Accident Analysis & Prevention, 2011, 43(2): 549-564.

[65] 汤泉. 人机协作拆卸序列规划方法研究[D]. 武汉: 武汉理工大学, 2018.

[66] Elmaraghy WH,Urbanic RJ. Modelling of Manufacturing Systems Complexity[J]. CIRP Annals,2003,52(1):363-366.

[67] Alex Cao, Keshav K,Chintamani, Abhilash, et al. NASA TLX: Software for assessing subjective mental workload[J]. Behavior Research Methods,2009,41:113–117.

[68] Rosemarie E. Yagoda. Development of the Human Robot Interaction Workload Measurement Tool (HRI-WM)[A]// Human Factors and Ergonomics Society 54th Annual Meeting[C].2010,304-308.

[69] Battiste V, Bortolussi M. Transport pilot workload—A comparison between two subjective techniques[A]// Human Factors and Ergonomics Society 32nd Annual Meeting[C]. 1988:150-154.

[70] 田卓. 基于多任务协同深度学习的疲劳检测系统研究[D]. 杭州: 杭州电子科技大学, 2017.

[71] Dot H S. Research on Vehicle-Based Driver Status,Performance Monitoring, Development,Validation,and Refinement of Algorithms For Detection of Driver Drowsiness[J].Administration Final Report Dot Hs National Highway Traffic Safety, 1994.

[72] Ferjani A,Ammar A,Pierreval H,et al.A simulation-optimization based heuristic for the online assignment of multi-skilled workers subjected to fatigue in manufacturing systems[J]. Computers & Industrial Engineering, 2017, 112: 663-674.

[73] Digiesi S, Kock A, Mummolo G, et al. The effect of dynamic worker behavior on flow line performance[J]. International Journal of Production Economics, 2009, 120(2): 368-377.

[74] Camillo Lugaresi, Jiuqiang Tang, Hadon Nash, et al. Mediapipe: A Framework for Building Perception Pipelines[J]. arXiv:1906.08172[cs],2019.

[75] Arqam M,Al-Nuimi, Ghassan J, et al. Face Direction Estimation based on Mediapipe Landmarks[C]// Proceedings of the 2021 7th International Conference on Contemporary Information Technology and Mathematics (ICCITM),2022.

[76] Martin Abadi, Paul Barham, Jianmin Chen, et al.Tensorflow: A system for large-scale machine learning[C]// Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), 2016,265–283.

[77] Tianqi Chen, Mu Li, Yutian Li, et al. Mxnet: A flexible and efficient machine learning library for heterogeneous distributed systems[J]. arXiv:1512.01274 [cs], 2015.

[78] Adam Paszke, Sam Gross, Soumith Chintala, et al. Automatic differentiation in pytorch[A]// NIPS[C],2017.

[79] Frank Seide,Amit Agarwal. Cntk: Microsoft’s open-source deep-learning toolkit[C]// Proceedings of 22nd ACM SIGKDD In-ternational Conference on Knowledge Discovery and DataMining, 2016,2135–2135.

[80] K. Huang, X. Liu, S. Fu, et al. A Lightweight Privacy Preserving CNN Feature Extraction Framework for Mobile Sensing[C]// Proceedings of the IEEE Transactions on Dependable and Secure Computing, 2019,1–1.

[81] Liuhao Ge, Hui Liang, Junsong Yuan, et al. Robust 3d hand pose estimation in single depth images: fromsingle-view cnn to multiview cnns[C]// Proceedings of the IEEE conference on computer vision and pattern recognition,2016,3593–3601.

[82] Liuhao Ge, Hui Liang, Junsong Yuan, et al. Robust 3d hand pose estimation from single depth images using multi-view cnns[C]// Proceedings of the IEEE Transactions on Image Processing, 2018,27(9):4422–4436.

[83] Liuhao Ge, Zhou Ren, Yuncheng Li, et al. 3d hand shape andpose estimation from a single rgb image[C]// Proceedings of The IEEE conference on computer vision and pattern recognition, 2019,10833–10842.

[84] Tu Chung-Jui,Chuang Li-Yeh,Chang Jun-Yang, et al. Feature Selection using PSO-SVM[C]// International Journal of Computer Science,2007,33:1.

[85] Fan Zhang,Valentin Bazarevsky,Andrey Vakunov,et al. Mediapipe Hands: On-device Real-time Hand Tracking[J]. 2019.

[86] Redmon J, Farhadi A. YOLO9000: Better, Faster, Stronger[J]. 2016.

[87] Redmon J, Farhadi A. YOLOv3: An Incremental Improvement[J]. 2018.

[88] Bochkovskiy A, Wang C Y, Liao H Y M. YOLOv4: Optimal Speed and Accuracy of Object Detection[J]. 2020.

[89] 霍凤财,杜颖,刘洋.人工蜂群算法及其应用[J]. 吉林大学学报(信息科学版),2016,34(04):468-476.

[90] 秦全德, 程适, 李丽, 等. 人工蜂群算法研究综述[J]. 智能系统学报, 2014, 9(02): 127-135.

[91] 王艳娇. 人工蜂群算法的研究与应用[D]. 哈尔滨: 哈尔滨工程大学, 2013.

[92] 马卫, 孙正兴. 基于精英蜂群搜索策略的人工蜂群算法[J]. 计算机应用 ,2014, 34(08): 2299-2305.

[93] 周新宇, 吴志健, 邓长寿, 等. 一种邻域搜索的人工蜂群算法[J]. 中南大学学报(自然科学版), 2015, 46(02): 534-546.

[94] Saeed Nezamivand Cheginia, Ahmad Bagheria, Farid Najafia. PSOSCALF A new hybrid PSO based on Sine Cosine Algorithm and Levy flight for solving optimization problems[J]. Applied Soft Computing. 2018, 73: 697-726.

[95] 邱剑锋. 人工蜂群算法的改进方法与收敛性理论的研究[D]. 安徽: 安徽大学, 2014.

[96] 宁爱平, 张雪英. 人工蜂群算法的收敛性分析[J]. 控制与决策, 2013, 28(10): 1554-1558.

中图分类号:

 TP242    

条码号:

 002000069695    

馆藏号:

 YD10000948    

馆藏位置:

 203    

备注:

 403-西院分馆博硕论文库;203-余家头分馆博硕论文库    

无标题文档

   建议浏览器: 谷歌 火狐 360请用极速模式,双核浏览器请用极速模式