-
单一方法难以同时兼顾图像的全局信息和局部信息,因此提出多方法融合的边缘提取方法。具体流程如图1所示,主要通过高斯滤波、K-Means聚类[23]、类间两两Otsu阈值分割[24]、分割后的图像进行或操作、图像取反保留最大连通域得到目标图像、提取目标边缘、Zernike矩亚像素边缘精定位[14]、输出亚像素目标边缘。
上述流程创新点如下:
(1)基于图像的全局和局部信息进行目标分割,针对图像边缘信息较复杂的情况具有较好的鲁棒性。基于全局信息对图像像素值进行聚类,对类间的像素值进行两两阈值分割是基于局部信息分割。
(2)将分割后的图像进行或操作后,由于金属表面的纹理和复杂光照导致图像出现反光和背光,出现孤立分割小区域,图像取反保留最大连通域得到目标图像,处理速度快,效果好。
(3) Zernike矩亚像素边缘定位可进一步提高目标的边缘位置,相关参数可根据边缘邻域像素值信息自适应更新。
-
实验对象为某线缆对接装置,见图2。包括供应端和接收端,供应端由执行机构、自适应机构和CCD相机等组成,CCD相机安装于自适应机构上。相机为海康威视的MV-CE100-30GM,分辨率为3856×2764,镜头为MVL-HF1228M-6MP。装置对接前需要知道供应端自适应机构和接收端的相对位姿,采用非合作目标位姿检测,故需要提取图2(c)虚线方框内孔的边缘,拟合出圆心进行位姿检测,圆心的世界坐标由上到下依次为(0,−85,−25)、(0,0,0)、(0,85,−25)。机械定位机构保证内孔均在图2(c)的虚线方框内。
图 2 设备原图。(a) 供应端;(b)接收端;(c)接收端上的特征
Figure 2. Original equipment image. (a) Actuator; (b) Receiver; (c) Features on the receiver
采集了85组图片,其中为了验证算法的正确性,在同一个曝光度采集了17个不同位置的图片,位置从远到近依次减少10 mm;为了验证算法的鲁棒性,在同一个位置采集5种不同曝光度(依次增强)下的图片,如图3所示。
-
图3中的材料为铝合金,铝合金在拍照时会出现反光的现象,导致出现亮光背景和暗光背景。为了解决这个问题,通过K-Means算法将图像的像素值分为3类,第1类是目标像素值,第2类是亮光的背景像素值,第3类是暗光的背景像素值,分类的结果如图4(b)所示。如果只将图像的像素值分为目标和背景两类,将会导致部分的背景被归类为目标类,致使边缘提取失败。
图 4 分类后阈值分割过程图片。(a)原图;(b)聚类后图像;(c)第1类和第3类像素值分割的结果;(d)第1类和第2类像素值分割的结果;(e)第2类和第3类像素值分割的结果;(f)图(d)和图(e)异或的结果;(g)图(c)、图(d)、图(e)相或运算的结果;(h)图(g)取反后根据连通域面积关系去掉伪连通域得到的目标
Figure 4. Image of threshold segmentation process after classification. (a) Original image; (b) Image after clustering; (c) Results of segmentation of class 1 and class 3 pixel values; (d) Results of segmentation of class 1 and class 2 pixel values; (e) Results of class 2 and class 3 pixel value segmentation; (f) The result of XOR of figure (d) and figure (e); (g) Results of figure (c), figure (d), figure (e) or operation; (h) The target obtained by removing the pseudo-connected components according to the area relationship of the connected components after inverting figure (g)
从图4(b)看出在边缘处附近存在着3类像素值,目标分割对阈值比较敏感。针对这种情况,将3类像素值进行两两阈值分割,剩下类的像素值视为0,使用Otsu算法进行分割。分割得到的3张二值图像进行或运算合并后得到初步的分割图像,其中包含着多个孤立小区域,见图4(g)。将图片取反后保留面积最大的连通域即是目标,见图4(h)所示。通过图4可以看出,分类后再Otsu阈值分割,可准确的将目标分割出来。
图5展示了各种分割算法效果对比,效果最好的是多方法融合的图像分割方法,其次是距离正则化水平集演化图像分割方法,其他方法均未有效地分割出目标。其他基于阈值分割算法基于全局信息寻找最佳阈值,多方法融合的图像分割方法先基于全局像素值进行分类,在不同类间进行阈值分割,既充分考虑全局信息又顾及目标边缘局部细节信息,故效果较好。多方法融合的图像分割方法思想是基于多阈值,但不是基于全局的阈值,而是不同类间的阈值。图5(c)中的多阈值是基于全局像素值进行多阈值分割,但由于金属表面的纹理易导致亮光背景和暗光背景,在目标边缘的像素值较复杂,未能考虑目标边缘的局部信息故其不能有效地分割出目标。距离正则化水平集演化图像分割方法需要正确的初始化,同时计算量较大,其分割一张图片需要226 s,而文中的方法只需要0.35 s,在实际应用中时效性太差。
图 5 各种分割方法对比。图片对应于图4中第1行第1张图的圆3。(a) Otsu; (b) 利用Otsu得到的全局直方图; (c) 多阈值; (d) 迭代阈值法;(e) 距离正则化水平集演化图像分割方法[18];(f) 文中方法
Figure 5. Comparison of various segmentation methods. The picture corresponds to circle 3 in the first picture in the first row in Fig. 4. (a) Otsu; (b) Global histogram threshold using Otsu's method; (c) Multilevel image thresholds using Otsu’s method; (d) Iterative threshold segmentation method; (e) DRLSE algorithm[18]; (f) Proposed method
-
阈值分割后的边缘为单像素,为了进一步提高位姿检测的精度,采用了参考文献[14]中Zernike矩进行边缘亚像素细化。图6为理想的亚像素边缘检测阶跃模型。图6(a)是原始边缘图像,图6(b)是旋转
$\phi $ 后的边缘图像。边缘两侧的灰度值分别为h和h+k,其中k为灰度差。边缘到原点的理论距离为l,l与x轴的夹角为$\phi $ 。图 6 亚像素边缘检测阶跃模型。(a)原始边缘图像; (b)旋转后边缘图像
Figure 6. Sub-pixel edge detection ideal step model. (a) Original edge image; (b) Rotated edge image
图7(a)是棋盘格标定板的一个角点。图7(b)是Zernike亚像素边缘和分类后阈值边缘的对比,Zernike亚像素边缘可修正分类后阈值分割的边缘。从图7(a)可以看出,由于相机像素间会有相互影响导致理想的边缘产生边缘模糊,可以看出边缘模糊在5个像素左右,所以Zernike掩膜的大小设置为
${\rm{5}} \times {\rm{5}}$ 。图 7 边缘细节。(a) 棋盘格标定板的一个角点; (b) Zernike亚像素边缘和分类后阈值边缘
Figure 7. Edge detail. (a) A corner of the checkerboard calibration board; (b) Zernike sub-pixel edges and thresholded edges after classification
根据参考文献[14]的方法计算出Zernike的边缘参数后,只有满足公式(1)的点是亚像素边缘上的点。
$$\left\{ {\begin{array}{*{20}{c}} {l < {l_{th}}} \\ {k > {k_{th}}} \end{array}} \right.$$ (1) 其中,
$l$ 应小于一个像素,参考文献[14]推荐${l_{th}} = \sqrt 2 {\rm{/2}}$ 。算法创新点之一:${k_{th}}$ 为粗边缘内3像素邻域的平均像素与外3像素邻域的平均像素差的一半。由于模板的放大效应[25],修正后的亚像素边缘坐标如下:$$\left[ {\begin{array}{*{20}{c}} {{x_s}} \\ {{y_s}} \end{array}} \right] = \left[ {\begin{array}{*{20}{c}} x \\ y \end{array}} \right] + \frac{{Nl}}{2}\left[ {\begin{array}{*{20}{c}} {\cos (\phi )} \\ {\sin (\phi )} \end{array}} \right]$$ (2) 式中:
$({x_s},{y_s})$ 是亚像素边缘坐标;$(x,y)$ 是粗边缘的坐标;N为Zernike模板的大小。 -
使用三个不共线的特征点求解位姿,特征点为三个内孔的拟合圆心,见图2(c)。机械结构定位保证了三个内孔会在虚线的方框内,方框在图像中的坐标是提前设定的,进行特征提取时,只需处理方框内的图像即可,可缩短计算时间和提高边缘的准确性。
在图8中,
${o_w} - {x_w}{y_w}{{\textit{z}}_w}$ 是世界坐标系,${o_c} - x{}_c{y_c}{{\textit{z}}_c}$ 是相机坐标系。点A、B、C在世界坐标系下的坐标已知,求出点A、B、C在相机坐标系下的坐标,即可求出世界坐标系与相机坐标系的相对位姿。根据余弦定理有:
$$\begin{split} \\ \left\{ {\begin{array}{*{20}{c}} {{O_c}{B^2} + {O_c}{C^2} - 2{O_c}B \cdot {O_c}C \cdot \cos \alpha = B{C^2}} \\ {{O_c}{A^2} + {O_c}{C^2} - 2{O_c}A \cdot {O_c}C \cdot \cos \beta = A{C^2}} \\ {{O_c}{A^2} + {O_c}{B^2} - 2{O_c}A \cdot {O_c}B \cdot \cos \beta = A{B^2}} \end{array}} \right. \end{split}$$ (3) 令
${O_c}A = x{O_c}C$ ,${O_c}B = y{O_c}C$ ,$p = 2\cos \alpha $ ,$q = 2\cos \beta $ ,$r = 2\cos \gamma $ ,$A{B^2} = v{O_c}{C^2}$ ,$B{C^2} = aA{B^2}$ ,$A{C^2} = bA{B^2}$ ;则公式(3)化简后得:$$\left\{ {\begin{array}{*{20}{c}} {(1 - a){y^2} - a{x^2} - yp + axyr + 1 = 0} \\ {(1 - b){x^2} - b{y^2} - xq + bxyr + 1 = 0} \end{array}} \right.$$ (4) 其中,
$p$ 、$q$ 、$r$ 、$a$ 、$b$ 是已知量,通过吴消元法即可求出$x$ 、$y$ ,则可以求出${O_c}A$ 、${O_c}B$ 、${O_c}C$ ,则$A = \overrightarrow {{O_c}a} \cdot \left\| {{O_c}A} \right\|$ ,$B = \overrightarrow {{O_c}b} \cdot \left\| {{O_c}B} \right\|$ ,$C = \overrightarrow {{O_c}c} \cdot \left\| {{O_c}C} \right\|$ 。
Monocular camera non-cooperative target extraction and pose detection
-
摘要: 针对单一方法难以兼顾图像全局和局部信息准确提取现场环境下的非合作目标边缘的难点,融合聚类等多种算法,提出一种边缘提取新方法。首先,根据图像像素值聚类,每2类间通过阈值进行分割得到1张二值图像;接着将二值图像进行或操作合并。将图像取反后保留最大面积的连通域得到目标分割图片,并提取目标边缘。最后,根据Zernike矩进行亚像素边缘计算。该边缘提取新方法具有较强的适应性,在实际的环境下均可快速有效提取出目标边缘。实验中的非合作目标为设备的三个内孔,用上述方法提取亚像素边缘后拟合出圆心,并用圆心进行相对位姿测量。实验结果表明,该方法鲁棒性强、精度高,最大的位置偏差为0.12 mm,垂直光轴方向姿态角的测量精度可达0.02°,其他两个姿态角的测量精度可达0.07°和0.08°。Abstract: Aiming at the difficulty of a single method to balance the global and local information to accurately extract the edges of non-cooperative targets in the field environment, combined clustering and other algorithms a new method of edge extraction was proposed. Firstly, according to the image pixel value clustering, each two categories were divided by threshold to obtain a binary image and the binary images were subjected to or operation. Then, after the image was inverted, the connected domain with the largest area was retained to obtain the target segmentation image, and the target edge was extracted. Finally, sub-pixel edge calculation based on Zernike moment was processed. The new edge extraction method had strong adaptability, and could quickly and effectively extract the target edge in the actual environment. The non-cooperative targets in the experiment were the three inner holes of the device. The sub-pixel edges were extracted by the above method, and then the center of the circle was fitted, and the center was used for relative pose measurement. The experimental results show that the method is robust and accurate. The maximum position deviation is 0.12 mm, the measurement precision of the attitude angle which is perpendicular to the optical can reach 0.02°, and the measurement precision of the other two attitude angles can reach 0.07° and 0.08°.
-
Key words:
- clustering /
- threshold segmentation /
- Zernike /
- non-cooperative target /
- pose detection
-
图 4 分类后阈值分割过程图片。(a)原图;(b)聚类后图像;(c)第1类和第3类像素值分割的结果;(d)第1类和第2类像素值分割的结果;(e)第2类和第3类像素值分割的结果;(f)图(d)和图(e)异或的结果;(g)图(c)、图(d)、图(e)相或运算的结果;(h)图(g)取反后根据连通域面积关系去掉伪连通域得到的目标
Figure 4. Image of threshold segmentation process after classification. (a) Original image; (b) Image after clustering; (c) Results of segmentation of class 1 and class 3 pixel values; (d) Results of segmentation of class 1 and class 2 pixel values; (e) Results of class 2 and class 3 pixel value segmentation; (f) The result of XOR of figure (d) and figure (e); (g) Results of figure (c), figure (d), figure (e) or operation; (h) The target obtained by removing the pseudo-connected components according to the area relationship of the connected components after inverting figure (g)
图 5 各种分割方法对比。图片对应于图4中第1行第1张图的圆3。(a) Otsu; (b) 利用Otsu得到的全局直方图; (c) 多阈值; (d) 迭代阈值法;(e) 距离正则化水平集演化图像分割方法[18];(f) 文中方法
Figure 5. Comparison of various segmentation methods. The picture corresponds to circle 3 in the first picture in the first row in Fig. 4. (a) Otsu; (b) Global histogram threshold using Otsu's method; (c) Multilevel image thresholds using Otsu’s method; (d) Iterative threshold segmentation method; (e) DRLSE algorithm[18]; (f) Proposed method
图 11 不同曝光度下不同尺寸掩膜的测量误差数据。(a) 不同尺寸掩膜的测量误差标准差图;(b) 不同尺寸掩膜的测量误差最大偏差图
Figure 11. The measurement error data of different size masks under different exposures. (a) The standard deviation of the measurement error of different size masks; (b) The maximum deviation of the measurement error of different size masks
-
[1] Qu Y P, Liu J Q, Hou W. Graphics design of cooperative targets on monocular vision high precision measurement [J]. Acta Optica Sinica, 2020, 40(13): 1315001. (in Chinese) doi: 10.3788/AOS202040.1315001 [2] Wang X J, Cao Y, Zhou K. Methods of monocular pose measurement based on planar objects [J]. Optics and Precision Engineering, 2017, 25(1): 274-280. (in Chinese) doi: 10.3788/OPE.20172501.0274 [3] Guan Y, Wang X J, Yin L, et al. Monocular position and pose measurement method based on surface topography of object [J]. Opto-Electronic Engineering, 2018, 45(1): 40-47. (in Chinese) doi: 10.12086/oee.2018.170522 [4] Chen G Y, Cheng Q L, Zhang J Y, et al. Multi-sensor measurement based position and pose adjustment method for automatic docking of spacecraft cabins [J]. Journal of Beijing University of Aeronautics and Astronautics, 2019, 45(6): 1232-1239. (in Chinese) doi: 10.13700/j.bh.1001-5965.2018.0593 [5] Su J D, Qi X H, Duan X S. Plane pose measurement method based on monocular vision and checkerboard target [J]. Acta Optica Sinica, 2017, 37(8): 0815002. (in Chinese) doi: 10.3788/AOS201737.0815002 [6] Wang Z Y, Li Y R, Hao R J, et al. Monocular vision pose measurement algorithm based on points feature [J]. Infrared and Laser Engineering, 2019, 48(5): 0517002. (in Chinese) doi: 10.3788/IRLA201948.0517002 [7] Faessler M, Mueggler E, Schwabe K, et al. A monocular pose estimation system based on infrared leds[C]//2014 IEEE international conference on robotics and automation (ICRA), 2014: 907-913. [8] Guo Y, Zhao D. Optimization design of non-coplanar target for pose measurement with monocular vision system [J]. Optik, 2017, 131: 72-78. doi: 10.1016/j.ijleo.2016.11.075 [9] Miao X K, Zhu F, Ding Q H, et al. Monocular vision pose measurement based on docking ring component [J]. Acta Optica Sinica, 2013, 33(4): 0412006. (in Chinese) doi: 10.3788/AOS201333.0412006 [10] Liu H, Wang Z, Wang B, et al. Pose determination of non-cooperative spacecraft based on multi-feature information fusion[C]//2013 IEEE International Conference on Robotics and Biomimetics (ROBIO), 2013: 1538-1543. [11] Zhang Xu, Wei Peng. Monocular vision calibration method of the stereo target for robot pose measurement [J]. Infrared and Laser Engineering, 2017, 46(11): 1117005. (in Chinese) doi: 10.3788/IRLA201746.1117005 [12] Chen Q J, Zhou Z Q, Liu C J, et al. Design of a vision calibration method of kinematic parameters for 6-DoF manipulator [J]. Journal of Tongji University(Natural Science), 2020, 48(7): 1023-1029. (in Chinese) [13] Yang Shangkun, Wang Yansong, Guo Hui. Sub-pixel measurement system of circle outer diameter based on Zernike moment[C]//2nd International Conference on Artificial Intelligence and Engineering Applications, 2017. [14] Ghosal S, Mehrotra R. Orthogonal moment operators for subpixel edge detection [J]. Pattern Recognition, 1993, 26(2): 295-306. doi: 10.1016/0031-3203(93)90038-X [15] Yang T, Zhao Q, Wang X, et al. Sub-pixel chessboard corner localization for camera calibration and pose estimation [J]. Applied Sciences, 2018, 8(11): 2118. doi: 10.3390/app8112118 [16] Arnab A, Zheng S, Jayasumana S, et al. Conditional random fields meet deep neural networks for semantic segmentation: Combining probabilistic graphical models with deep learning for structured prediction [J]. IEEE Signal Processing Magazine, 2018, 35(1): 37-52. doi: 10.1109/MSP.2017.2762355 [17] Takikawa T, Acuna D, Jampani V, et al. Gated-scnn: Gated shape cnns for semantic segmentation [C]//Proceedings of the IEEE International Conference on Computer Vision. 2019: 5229-5238. [18] Li C, Xu C, Gui C, et al. Distance regularized level set evolution and its application to image segmentation [J]. IEEE Transactions on Image Processing, 2010, 19(12): 3243-3254. doi: 10.1109/TIP.2010.2069690 [19] Getreuer P. Chan-vese segmentation [J]. Image Processing On Line, 2012, 2: 214-224. doi: 10.5201/ipol.2012.g-cv [20] Dhanachandra N, Manglem K, Chanu Y J. Image segmentation using K-means clustering algorithm and subtractive clustering algorithm [J]. Procedia Computer Science, 2015, 54: 764-771. doi: 10.1016/j.procs.2015.06.090 [21] Thilagamani S, Shanthi N. A survey on image segmentation through clustering [J]. International Journal of Research and Reviews in Information Sciences, 2011, 1(1): 14-17. [22] Han J, Yang C, Zhou X, et al. A new multi-threshold image segmentation approach using state transition algorithm [J]. Applied Mathematical Modelling, 2017, 44: 588-601. doi: 10.1016/j.apm.2017.02.015 [23] Arthur D, Vassilvitskii S. K-means++: The advantages of careful seeding[C]//Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, 2007: 1027-1035. [24] Otsu N. A threshold selection method from gray-level histograms [J]. IEEE Transactions on Systems, Man, and Cybernetics, 1979, 9(1): 62-66. doi: 10.1109/TSMC.1979.4310076 [25] Li J Q, Wang J W, Chen S B, et al. Improved algorithm of subpixel edge detection using Zernike orthogonal moments [J]. Optical Technique, 2003(4): 500-503. (in Chinese) doi: 10.3321/j.issn:1002-1582.2003.04.036 [26] Banno A. A P3P problem solver representing all parameters as a linear combination [J]. Image and Vision Computing, 2018, 70: 55-62. doi: 10.1016/j.imavis.2018.01.001