罗陆锋, 邹湘军, 叶敏, 杨自尚, 张丛, 朱娜, 王成琳. 基于双目立体视觉的葡萄采摘防碰空间包围体求解与定位[J]. 农业工程学报, 2016, 32(8): 41-47. DOI: 10.11975/j.issn.1002-6819.2016.08.006
    引用本文: 罗陆锋, 邹湘军, 叶敏, 杨自尚, 张丛, 朱娜, 王成琳. 基于双目立体视觉的葡萄采摘防碰空间包围体求解与定位[J]. 农业工程学报, 2016, 32(8): 41-47. DOI: 10.11975/j.issn.1002-6819.2016.08.006
    Luo Lufeng, Zou Xiangjun, Ye Min, Yang Zishang, Zhang Cong, Zhu Na, Wang Chenglin. Calculation and localization of bounding volume of grape for undamaged fruit picking based on binocular stereo vision[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2016, 32(8): 41-47. DOI: 10.11975/j.issn.1002-6819.2016.08.006
    Citation: Luo Lufeng, Zou Xiangjun, Ye Min, Yang Zishang, Zhang Cong, Zhu Na, Wang Chenglin. Calculation and localization of bounding volume of grape for undamaged fruit picking based on binocular stereo vision[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2016, 32(8): 41-47. DOI: 10.11975/j.issn.1002-6819.2016.08.006

    基于双目立体视觉的葡萄采摘防碰空间包围体求解与定位

    Calculation and localization of bounding volume of grape for undamaged fruit picking based on binocular stereo vision

    • 摘要: 无损收获是采摘机器人的研究难点之一,葡萄采摘过程中容易因机械碰撞而损伤果实,为便于机器人规划出免碰撞路径,提出一种基于双目立体视觉的葡萄包围体求解与定位方法。首先通过图像分割获得葡萄图像质心及其外接矩形,确定果梗感兴趣区域并在该区域内进行霍夫直线检测,通过寻找与质心距离最小的直线来定位果梗上的采摘点,运用圆检测法获取外接矩形区域内果粒的圆心和半径。然后运用归一化互相关的立体匹配法求解采摘点和果粒圆心的视差,利用三角测量原理求出各点的空间坐标。最后以采摘点的空间坐标为原点构建葡萄空间坐标系,求解葡萄最大截面,再将该截面绕中心轴旋转360°得到葡萄空间包围体。试验结果表明:当深度距离在1 000 mm以内时,葡萄空间包围体定位误差小于5 mm,高度误差小于4.95%,最大直径误差小于5.64%,算法时间消耗小于0.69 s。该研究为葡萄采摘机器人的防损采摘提供一种自动定位方法。

       

      Abstract: Abstract: Undamaged picking is one of difficulties in automatic harvesting robots. Since grape is a cluster growing fruit and its pericarp and sarcocarp are weak, so grape is easy to be collided and damaged by the manipulator and end-effector when they approach to pick a candidate grape. To plan a collision-free path, a calculation and localization method for bounding volume of grape based on binocular stereo vision was presented. The vision system was consisted of two MV-VD120SC color cameras and the baseline distance of two cameras was 50 mm. Firstly, the binocular stereo vision system was calibrated by using a calibration plate ordered from MVTec Software GmbH (Germany), and subsequently the images captured by two cameras were rectified. Secondly, the grape cluster region was acquired by segmenting the rectified left image using an adaptive threshold method based on H-I color component. The exterior rectangle and barycenter of the region were extracted. The region of interest of peduncle was determined according to those extracted geometric information, and subsequently the picking point on peduncle was calculated out by combining the Hough line detection and the minimum distance restraint between barycenter and the detected line. The center and radius of grape berries were acquired using circle regression within the exterior rectangle. To accelerate circle regression and enhance the accuracy of berries recognition, an adaptive predication model of the berry radius in images captured at various distances was built through power multiplication method, and two rules were built to eliminate these redundant circles that were produced by circle regression. Thirdly, the disparity of the picking point and the center of berries between left and right images were calculated by stereo matching method based on similarity function of normalized correlation coefficient, and subsequently three-dimensional coordinates of picking point and center of berries were extracted by using the triangulation principle. Three-dimensional virtual grape cluster was rendered by using OpenGL functions. Fourthly, the coordinate system XYZ of grape cluster was built, which defined the picking point as origin and took vertical downward Y axis as grape of center axis. The distance L between origin and each detected berry under XZ plane was calculated, the Y-L coordinate system that was used to solve the maximum section of grape cluster was built, and the maximum contour section of grape cluster was obtained through seeking convex polygon, the bounding volume of grape acquired by rotating the section 360 degree around the center axis. Finally, to validate the performance of the proposed method, thirty pairs of images were captured at various distances within 600-1 100 mm using binocular stereo vision system in the Tianjin Chadian Grapes Science Park. The localization and size accuracy of grape bounding volume and the elapsed time of algorithm were tested. The test results indicated that the localization error of grape bounding volume was less than 5 mm, the relative error of height and maximum diameter was less than 4.95% and 5.64%, respectively, and the elapsed time of algorithm was less than 0.69s when the depth was less than 1 000 mm, which showed that the proposed method can prevent grape from damaging during grape picking. However, the localization accuracy was not satisfactory when multiple grapes were overlapping each other or the occlusion was serious, so further studies should focus on the localization of multiple overlapping grapes and occluded grapes.

       

    /

    返回文章
    返回