-
快速三维测量使用二值离焦技术来提高测量速度,二值方波凭借其获取简易以及一致性较好等一系列优点,在快速三维测量中得到广泛应用。三步相移算法因其所需投影光栅帧数较少,实际快速测量可通过离焦投影三帧相移二值方波光栅实现[7]。具体测量流程如图1所示。
图1(a)表示相机实际采集得到三帧相移光栅图像,具体可描述为[3]:
$$I_n^C(x,y) = {B_0}(x,y) + \sum\limits_{k = 1}^\infty {\left\{ {{B_k}(x,y)\cos [k{\phi _n}(x,y)]} \right\}} $$ (1) 式中:n = 1,2,···,N表示相移步数;
$(x,y)$ 为像素坐标;${B_0}$ 为背景光强;${B_k}$ 表示第k阶高次谐波的幅度信息;${\phi _n}$ 为耦合相位,可描述为:$${\phi _n}(x,y) = \phi (x,y) + {\delta _n}$$ (2) 式中:
$\phi $ 是理想相位信息;${\delta _n}$ 为第n帧条纹相移量,具体为:$${\delta _n} = 2\pi (n - 1)/N$$ (3) 为了简化起见,下文省略像素坐标
$(x,y)$ 。图1(b)为采用相移算法计算所得真实相位
${\phi ^C}$ ,通常采用最小二乘方法实现[6]:$${\phi ^C} = \arctan \left[ {\frac{{ - \displaystyle\sum\limits_{n = 1}^\infty {(I_n^C\sin {\delta _n})} }}{{\displaystyle\sum\limits_{n = 1}^\infty {(I_n^C\cos {\delta _n})} }}} \right]$ .$ (4) 由于采集条纹中含有高次谐波,真实相位信息中会引入相位误差形如[3]:
$$\Delta {\phi _N} = {\phi ^C} - \phi = \arctan \left\{ {\frac{{\displaystyle\sum\limits_{m = 1}^\infty {\left[ {B_{mN}^{ - 1}\sin (mN\phi )} \right]} }}{{{B_1} + \displaystyle\sum\limits_{m = 1}^\infty {\left[ {B_{mN}^{ + 1}\cos (mN\phi )} \right]} }}} \right\}$$ (5) 式中:
$B_{mN}^{ - 1}{\rm{ = }}B_{mN{\rm{ + }}1}^{} - B_{mN - 1}^{}$ ;$B_{mN}^{{\rm{ + }}1}{\rm{ = }}B_{mN{\rm{ + }}1}^{}{\rm{ + }}B_{mN - 1}^{}$ 。该类型相位误差会给实际三维测量带来不可忽略的测量误差。上述过程求解得到的相位信息
${\phi ^C}$ 是在(−π,π]范围内的一系列呈锯齿状不连续分布相位。为了获得物体的绝对相位$\varPhi$ ,需要对${\phi ^C}$ 进行相位展开,相位解包裹的方法有时域法和空域法,如图1(c)文中使用格雷编码法展开包裹相位[10]:$$\varPhi {\rm{ }} = {\phi ^C}{\rm{ }} + K{\rm{ }} \times {\rm{ }}2{\text{π}} $$ (6) 式中:
$K$ 是对图像进行解码确定的相移条纹的级次信息,如图1(d)。最后,利用标定后的系统参数就可以从绝对相位中得到图1(f)的物体的三维信息[11-13]。 -
旨在轻微离焦程度下减少以致削弱高次谐波引入的测量误差,实现精确相位获取,进而实现快速精确三维测量,提出了基于深度学习精确相位获取的离焦投影三维测量方法,所提方法中使用深度学习实现精确相位获取的流程如图2所示。
图 2 精确相位获取的原理图。(a) 有噪声的相位图像;(b) 提出的PDNet;(c) 使用所提方法得到的结果
Figure 2. Schematic of accurate phase acquisition. (a) Phase with noise;(b) proposed PDNet;(c) result by the proposed method
与传统快速三维测量方法相比,所提方法在测量过程中引入基于深度学习的PDNet算法对包裹相位进行计算以得到精确相位信息。引入的PDNet算法构建含噪声相位
${\phi ^C}$ 到近似无噪声理想相位$\phi $ 的端到端方式训练模型,训练过程中算法每次迭代计算出的精确相位$\phi _i^{out}$ 与理想相位图像$\phi $ 进行残差学习将产生的误差返回到网络中,使网络不断迭代,得到一个最优的网络参数,使其具有较好的去噪能力同时对不同程度的噪声具有较好的鲁棒性。文中使用的误差函数如下式:$$Loss = \frac{1}{m}\sum {\left\| {\phi _i^{{\rm{out}}} - \phi } \right\|^2}$$ (7) 式中:
$\phi _i^{{\rm{out}}}$ 为PDNet训练过程中每次迭代计算得到的相位图像,i为迭代次数,$\phi $ 为理想相位图像;m为相位图像中有效像素个数。最终,训练完成后利用得到的PDNet模型能够对不同场景下获取的相位图像${\phi ^C}$ 进行去噪,以得到测量所需精确相位图像${\phi ^{{\rm{out}}}}$ 。投影设备轻微离焦程度下,实际测量过程中仅需30次以下高次谐波影响,根据方波误差分布规律,投影十五步高频二值方波条纹,即可得到理想相位
$\phi $ 。对应的输入含噪声相位图像${\phi ^C}$ 即可选取三步相移算法求解得到。使用训练后得到的PDNet模型对输入相位图像${\phi ^C}$ 进行运算即可得到精确相位图像${\phi ^{{\rm{out}}}}$ ,精确相位图像${\phi ^{{\rm{out}}}}$ 与理想相位图像$\phi $ 之间存在的相位误差可用$\Delta \phi _{}^{{\rm{out}}}$ 表示,可简单用数学描述为:$$\Delta \phi _{}^{{\rm{out}}}{\rm{ }} = \phi - {\phi ^{{\rm{out}}}}$$ (8) 据此式即可进行相位误差评估。依据去噪后的精确相位
${\phi ^{{\rm{out}}}}$ 即可得到物体的准确三维信息。如图2所示,PDNet算法训练完成后使用模型就能够在测量过程中将图2(a)含有噪声的相位图像
${\phi ^C}$ 作为输入层计算,得到去噪后的精确相位图像${\phi ^{{\rm{out}}}}$ ,如图2(c)所示。图2(b)为引入的PDNet,具体结构见图3。如图3所示,对于网络结构的设计,文中借鉴了高效残差分解网络(Efficient Residual Factorized Network,ERFNet)[14],在其基础上对网络结构进行了改进,增加了跳层连接并去掉了池化层,以更好地匹配文中的图像去噪任务。因为增加跳层可以更好地进行残差学习,利于噪声的去除,而池化层则可能导致图像丢失细节信息,内部的网络层均由特征图所构成。编码过程和解码过程都含有多尺寸的卷积层与相关的策略层,其主要模块及相关参数设置如表1所示。表1中,“Out-F”为输出图层通道数,“Out-Res”为假设输入尺寸为1 024×512时各层的输出分辨率。
表 1 网络主要模块及参数
Table 1. Main network modules and parameters
Type Out-F Out-Res ENCODER 1 Downsampler block 16 512×256 2 Downsampler block 64 256×128 3−7 5 × Non-bt-1D 64 256×128 8 Downsampler block 128 128×64 9 Non-bt-1D(dilated 2) 128 128×64 10 Non-bt-1D(dilated 4) 128 128×64 11 Non-bt-1D(dilated 8) 128 128×64 12 Non-bt-1D(dilated 16) 128 128×64 13 Non-bt-1D(dilated 2) 128 128×64 14 Non-bt-1D(dilated 4) 128 128×64 15 Non-bt-1D(dilated 8) 128 128×64 16 Non-bt-1D(dilated 16) 128 128×64 DECODER 17 Deconvolution (unsampling) 64 256×128 18−19 2 × Non-bt-1D 64 256×128 20 Deconvolution (unsampling) 16 512×256 21−22 2 × Non-bt-1D 16 512×256 23 Deconvolution (unsampling) C 1 024×512 该网络模型结构主要包含三个模块:Downsampler block,Non-bt-1D(dilated),Deconvolution (unsampling):
(1) Downsampler block
该模块包括:卷积核大小3×3的卷积层,并根据参数设置输入及输出的特征图通道数,用于获取噪声图的特征分布;BN层(Batch Normalization),可防止梯度消失或爆炸;ReLu激活层,也可克服梯度消失的问题并通过特征稀疏性加快网络速度。
(2) Non-bt-1D(dilation rate)
该模块包括:两层卷积核大小3×1的卷积层,两层卷积核大小1×3的卷积层,并可设置卷积核的间隔数量(dilation rate),形成空洞卷积以增大感受野,两层BN层,还包括dropout策略,即在前向传播的时候,让某个神经元的激活值以一定的概率停止工作,使得它不会太依赖某些局部的特征,这样可以使模型泛化性更强。
(3) Deconvolution (unsampling)
此模块主要包括一层卷积层、BN层和ReLu激活层,该模块主要目的是为了对特征进行上采样,最终恢复出与原图相同尺寸的目标输出图。
Defocus projection three-dimensional measurement based on deep learning accurate phase acquisition
-
摘要: 数字光栅投影三维测量技术,通过离焦投影二值光栅条纹,生成三维测量所需正弦光栅条纹,能够实现超高投影速度,在快速三维测量领域具有极大潜力。但是,二值光栅条纹不可避免地包含高次谐波,导致计算所得相位包含相位误差,进而降低了快速三维测量精度。提出了一种基于深度学习精确相位获取的离焦投影三维测量方法,通过构建含噪声相位到精确相位的端到端深度卷积神经网络,降低高次谐波引入的相位误差,进而实现快速精确三维测量。首先,以理论分析证明所提方法的可行性,并以仿真和实验进一步验证了所提方法的有效性和精确性。与现有快速三维测量方法相比,所提方法在保证测量速度的同时保证测量精度。Abstract: The digital fringe projection three-dimensional (3D) measurement technology can generate a sinusoidal fringe pattern for 3D measurement by defocusing a binary fringe pattern. It can achieve extremely high projection speed and has great potential in the field of high-speed 3D measurement. However, the binary fringe pattern inevitably contains higher-order harmonics, resulting in a phase error introduced into the calculated phase, thereby reducing the accuracy of high-speed 3D measurement. A 3D measurement method for defocused projection based on deep learning accurate phase acquisition was proposed. The image feature processing capability based on deep learning algorithm can remove the phase errors introduced by higher-order harmonics. An end-to-end deep convolutional neural network from noise phase to precise phase was constructed by this method and the phase error introduced by higher-order harmonics was reduced. Finally, high-speed and accurate 3D measurement could be achieved by this method. Firstly, the theoretical analysis proved the feasibility of the proposed method. Then, simulation and experiments were performed to further verify the effectiveness and accuracy of the proposed method. Compared with the existing high-speed 3D measurement methods, the proposed method can ensure measurement speed while ensuring measurement accuracy.
-
Key words:
- 3D measurement /
- defocused projection /
- deep learning /
- accurate phase
-
图 4 仿真结果。(a) 有噪声的相位;(b) 使用所提方法得到的结果;(c) 图4(a)和4(b)与理想相位对比第100行细节信息;(d) 图4(b)和4(c)第100行误差曲线
Figure 4. Simulation results. (a) Phase map with noise;(b) result by the proposed method;(c) comparison of the phase in the 100th column from Fig. 4(a), 4(b) and the ground truth;(d) plotting error curves of the 100th column of Fig. 4(b) and 4(c)
图 6 实验结果。(a) 有噪声的相位;(b) 使用所提方法得到的结果;(c) 图6(a)和6(b)第100行细节信息与理想相位对比;(d) 图6(b)和6(c)第100行误差曲线
Figure 6. Experiment results. (a) Phase map with noise;(b) result by the proposed method;(c) comparison of the phase in the 100th column from Fig. 6(a), 6(b) and the ground truth;(d) plotting error curves of the 100th column of Fig. 6(b) and 6(c)
图 9 重建效果对比。(a) 使用所提方法处理前的重建结果;(b) 理想情况下的重建结果;(c) 使用所提方法处理后的重建结果;(d) 重建误差曲线
Figure 9. Comparison of reconstruction results. (a) Reconstruction results before processing using the proposed method;(b) ideal reconstruction results;(c) reconstruction result by the proposed method;(d) reconstruction error curve
表 1 网络主要模块及参数
Table 1. Main network modules and parameters
Type Out-F Out-Res ENCODER 1 Downsampler block 16 512×256 2 Downsampler block 64 256×128 3−7 5 × Non-bt-1D 64 256×128 8 Downsampler block 128 128×64 9 Non-bt-1D(dilated 2) 128 128×64 10 Non-bt-1D(dilated 4) 128 128×64 11 Non-bt-1D(dilated 8) 128 128×64 12 Non-bt-1D(dilated 16) 128 128×64 13 Non-bt-1D(dilated 2) 128 128×64 14 Non-bt-1D(dilated 4) 128 128×64 15 Non-bt-1D(dilated 8) 128 128×64 16 Non-bt-1D(dilated 16) 128 128×64 DECODER 17 Deconvolution (unsampling) 64 256×128 18−19 2 × Non-bt-1D 64 256×128 20 Deconvolution (unsampling) 16 512×256 21−22 2 × Non-bt-1D 16 512×256 23 Deconvolution (unsampling) C 1 024×512 -
[1] Zhang S. Recent progresses on real-time 3D shape measurement using digital fringe projection techniques [J]. Optics and Lasers in Engineering, 2010, 48(2): 149−158. doi: 10.1016/j.optlaseng.2009.03.008 [2] Hong Ziming, Ai Qingsong, Chen Kun. High precise 3D visual measurement based on fiber laser [J]. Infrared and Laser Engineering, 2018, 47(8): 0803011. (in Chinese) doi: 10.3788/IRLA201847.0803011 [3] Zheng D, Da F, Kemao Q, et al. Phase error analysis and compensation for phase shifting profilometry with projector defocusing [J]. Applied Optics, 2016, 55(21): 5721. doi: 10.1364/AO.55.005721 [4] Dai Meiling, Yang Fujun, He Xiaoyuan. Three-dimensional shape measurement of objects with discontinuities by dual-frequency color fringe projection [J]. Optics and Precision Engineering, 2013, 21(1): 11−16. (in Chinese) [5] Su X, Chen W. Fourier transform profilometry: a review [J]. Optics and Lasers in Engineering, 2001, 35(5): 263−284. doi: 10.1016/S0143-8166(01)00023-9 [6] Zuo C, Feng S, Huang L, et al. Phase shifting algorithms for fringe projection profilometry: A review [J]. Optics and Lasers in Engineering, 2018, 109: 23−59. doi: 10.1016/j.optlaseng.2018.04.019 [7] Song Z. Flexible 3D shape measurement using projector defocusing: extended measurement range [J]. Optics Letters, 2010, 35(7): 934−6. doi: 10.1364/OL.35.000934 [8] Zuo C, Chen Q, Feng S, et al. Optimized pulse width modulation pattern strategy for three-dimensional profilometry with projector defocusing [J]. Applied Optics, 2012, 51(19): 4477−4490. doi: 10.1364/AO.51.004477 [9] Zhao Liwei, Da Feipeng, Zheng Dongliang. Method for binary grating generation using defocused projection for three-dimensional measurement [J]. Acta Optica Sinica, 2016, 36(8): 0812005. (in Chinese) doi: 10.3788/AOS201636.0812005 [10] Zheng D, Da F. Self-correction phase unwrapping method based on gray-code light [J]. Optics and Lasers in Engineering, 2012, 50(8): 1130−1139. doi: 10.1016/j.optlaseng.2012.01.019 [11] Zhang Lei, Jiao Xiaoxue, Zhou Liqiu, et al. Three-dimensional shape acquisition method by integral imaging based on corresponding points [J]. Chinese Optics, 2015, 8(1): 45−50. (in Chinese) doi: 10.3788/co.20150801.0045 [12] Sun Riming, Lin Tingting, Ji Lin, et al. A general linear imaging modeling method for space unstability targets and parameter optimization of linear laser radar [J]. Optics and Precision Engineering, 2018, 26(6): 1524−1532. (in Chinese) doi: 10.3788/OPE.20182606.1524 [13] An Dong, Chen Li, Ding Yifei, et al. Optical system model and calibration of grating projection phase method [J]. Chinese Optics, 2015, 8(2): 248−254. (in Chinese) doi: 10.3788/co.20150802.0248 [14] Romera E, Alvarez J M, Bergasa L M, et al. ERFNet: Efficient residual factorized ConvNet for real-time semantic segmentation [J]. IEEE Transactions on Intelligent Transportation Systems, 2018, 19(1): 263−272. doi: 10.1109/TITS.2017.2750080