无规则扰动状态下柑橘果实在线目标检测与快速定位
作者:
作者单位:

作者简介:

通讯作者:

中图分类号:

S225;TP242;TP391.41

基金项目:

重庆市科委产业化重点专项(cstc2018jszx-cyzdX0051)


Online target detection and fast localisation of citrus fruits in irregularly disturbed state
Author:
Affiliation:

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    柑橘采摘机器人连续采摘过程中因各种因素会引起其他待采柑橘无规则扰动,扰动状态下的柑橘无法在线快速准确检测与定位,影响机器人采摘效率。针对此问题,该研究提出一种基于改进YOLOv5s+DeepSORT算法的扰动柑橘在线目标检测与快速定位方法。首先在YOLOv5骨干网络中融入卷积注意力机制(convolutional block attention module, CBAM),提升模型对复杂目标的检测能力;用SIoU(scalable intersection over union)损失函数增强预测框与标定框之间的方向匹配,提升回归收敛速度。其次在DeepSORT算法中改进目标重识别网络(re-identification, ReID),增强网络特征提取能力,提升目标跟踪准确度与精度;在算法中融入Count计数机制,实时反馈每个扰动柑橘跟踪帧数,并改进算法实现对预测坐标值进行实时更新,提升预测准确率。最后结合深度相机排除背景柑橘影响并限制每次跟踪目标数目为3个,提升扰动柑橘预测定位速度。试验结果表明,与原算法相比,改进YOLOv5s算法的准确度、平均检测精度分别提升3.9、1.1个百分点,检测速率69.3帧/s。改进DeepSORT算法的跟踪准确度、跟踪精度分别提升9.2、5.4个百分点,ID(identity)切换次数减少32次。当预测定位时间为3 s时,定位平均准确度为81.9%,在试验室进行模拟试验,将盆栽柑橘果实沿不同方位随机摆动,摆幅约10 cm,单个柑橘平均抓取时间为12.8 s,比未使用改进算法缩短5.6 s,效率提升30.4 %。该研究可为扰动状态下的柑橘快速采摘提供技术支持。

    Abstract:

    Picking robots have been widely used for citrus harvesting in recent years. However, the rest citrus during continuous picking can be irregularly disturbed by the wind, robot force, and the load weight of bearing branches under the natural environment. The citrus in the disturbed state cannot be rapidly and accurately detected, and then localized online, leading to the low efficiency of automatic robotic picking. In this study, online target detection and rapid localization were proposed using improved YOLOv5s+DeepSORT. The position of citrus at rest was predicted using the motion-tracking trajectory of disturbed citrus within a short period of time. The coordinates of the citrus were then obtained rapidly. Firstly, the CBAM (Convolutional Block Attention Module) attention mechanism was added to the YOLOv5s network, in order to detect the small and occluded targets. The SIoU loss function was used to enhance the direction matching between the prediction and the calibration frame, in order to improve the convergence speed of regression. Secondly, the target re-identification network was improved in the DeepSORT more suitable for the feature extraction of citrus targets. The feature extraction of the network was enhanced to improve the tracking performance on the disturbed citrus; The Count counter was used to accumulate the number of tracking frames in each citrus for an optimal target. Since the disturbance of the rest citrus was progressively propagated over time, the localization prediction and picking were only for targets with optimal tracking trajectories at a time. The real-time updating was realized in real time. Finally, the values of the depth camera were combined within the critical distance range, excluding the influence of background citrus on the detection speed. The number of tracking targets each time was limited to effectively improve the tracking speed of disturbed citrus. The experimental results show that the P (precision) and mAP (average detection accuracy) of improved YOLOv5s were improved by 3.9 and 1.1 percentage points, respectively, with a detection rate of 69.3 frames per second. The MOTA (Multi-Object Tracking Accuracy) and MOTP (Multi-Object Tracking Precision) of the improved DeepSORT were improved by 9.2 and 5.4 percentage points, respectively, whereas, the average number of ID (identity) switching times of targets was reduced by 32 times. Grasping experiments were conducted in the laboratory, in which the citrus was randomly swung along different orientations with an amplitude of about 10 cm. When the predicted localization time was 1, 2, 3, 5, 7, and 10 s, the average precision values of disturbed citrus localization were 21.3%, 53.0%, 81.9%, 83.7%, 86.1%, and 94.9%, respectively. The citrus picking test was conducted with the citrus localization time of 3 s. The average grabbing time for each citrus was 12.8 s, which was 5.6 s shorter than that without the optimization. The efficiency was improved by 30.4%. This finding can provide technical support and references for citrus picking in disturbed states.

    参考文献
    相似文献
    引证文献
引用本文

娄欢欢,李光林,付兴兰,李丽,王旭,黄伟东,付泰戈.无规则扰动状态下柑橘果实在线目标检测与快速定位[J].农业工程学报,2024,40(18):155-166. DOI:10.11975/j. issn.1002-6819.202404120

LOU Huanhuan, LI Guanglin, FU Xinglan, LI Li, WANG Xu, HUANG Weidong, FU Taige. Online target detection and fast localisation of citrus fruits in irregularly disturbed state[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE),2024,40(18):155-166. DOI:10.11975/j. issn.1002-6819.202404120

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2024-04-17
  • 最后修改日期:2024-06-06
  • 录用日期:
  • 在线发布日期: 2024-09-11
  • 出版日期:
文章二维码
您是第位访问者
ICP:京ICP备06025802号-3
农业工程学报 ® 2024 版权所有
技术支持:北京勤云科技发展有限公司