Depth estimation technique of sequence image based on deep learning

Liang Xinkai; Song Chuang; Zhao Jiajia

doi:10.3788/IRLA201948.S226002

Aiming at the weak generalization of single-frame image depth estimation, a depth estimation technique based on deep learning was proposed, which used deep convolutional neural network as the basic framework and combined the epipolar geometry constraints to construct the end-to-end mapping from sequence images to the depth information, enabling unsupervised depth estimation that only relied on sequence image information. At the same time, a kind of loss function based on three-dimensional geometric information of the scene was constructed, and the original loss function based on the re-projection error between images was discarded to improve the robustness of the algorithm. Finally, the accuracy and precision of the algorithm were verified by the open source database. At the same time, the generalization of the algorithm was verified by the infrared image dataset, which laid a foundation for the military application.

Depth estimation technique of sequence image based on deep learning

1. Science and Technology on Complex System Control and Intelligent Agent Cooperation Laboratory,Beijing Electro-Mechanical Engineering Institute,Beijing 100074,China

Received Date: 2019-05-12

Rev Recd Date: 2019-06-20

Publish Date: 2019-09-30

Key words:

Abstract: Aiming at the weak generalization of single-frame image depth estimation, a depth estimation technique based on deep learning was proposed, which used deep convolutional neural network as the basic framework and combined the epipolar geometry constraints to construct the end-to-end mapping from sequence images to the depth information, enabling unsupervised depth estimation that only relied on sequence image information. At the same time, a kind of loss function based on three-dimensional geometric information of the scene was constructed, and the original loss function based on the re-projection error between images was discarded to improve the robustness of the algorithm. Finally, the accuracy and precision of the algorithm were verified by the open source database. At the same time, the generalization of the algorithm was verified by the infrared image dataset, which laid a foundation for the military application.

HTML

Reference (14)

[1]	Lin Jinhua, Wang Yanjie. Three-dimentional reconstruction of semantic scene based on RGB-D map[J]. Optics and Precision Engineering, 2018, 26(5):1231-1241. (in Chinese)林金花, 王延杰. 三维语义场景复原网络[J]. 光学精密工程, 2018, 26(5):1231-1241.
[2]	Liu Bo, Xu Tingfa, Li Xiangmin, et al. Adaptive context-aware correlation filter tracking[J]. Chinese Optics, 2019, 12(2):265-273. (in Chinese)刘波, 许廷发, 李相民, 等. 自适应上下文感知相关滤波跟踪[J]. 中国光学, 2019, 12(2):265-273.
[3]	Li Yu, Liu Xueying, Zhang Hongqun, et al. Optical remote sensing image retrieval based on convolutional neural networks[J]. Optics and Precision Engineering, 2018, 26(1):200-207. (in Chinese)李宇, 刘雪莹, 张洪群, 等. 基于卷积神经网络的光学遥感图像检索[J]. 光学精密工程, 2018, 26(1):200-207.
[4]	Fan Xiaoting, Li Yi, Luo Xiaowei, et al. Depth estimation based on light field structure characteristic and multiview matching[J]. Infrared and Laser Engineering, 2019, 48(5):0524001. (in Chinese)范晓婷, 李奕, 罗晓维, 等. 基于光场结构特性与多视点匹配的深度估计[J]. 红外与激光工程, 2019, 48(5):0524001.
[5]	Eigen D, Puhrsch C, Fergu R. Depth map prediction from a single image using a multi-scale deep network[C]//Proc Conf Neural Information Processing Systems, 2014.
[6]	Eigen David, Rob Fergus. Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture[C]//The IEEE International Conference on Computer Vision, 2015.
[7]	Laina I, Rupprecht C, Belagiannis V. Deeper depth prediction with fully convolutional residual networks[C]//International Conference on 3D Vision, 2016.
[8]	Godard C, Aodha O M, Brostow G J. Unsupervised monocular depth estimation with left-right consistency[C]//The IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[9]	Zhou T, Brown M, Snavely N, et al. Unsupervised learning of depth and ego-motion from video[C]//The IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[10]	Ummenhofer B, Zhou H, Uhrig J, et al. DeMoN:Depth and motion network for learning monocular stereo[C]//The IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[11]	Huang P, Matzen K, Kopf J, et al. DeepMVS:Learning multi-view stereopsis[C]//The IEEE Conference on Computer Vision and Pattern Recognition, 2018.
[12]	Kendall A, Martirosyan H, Dasgupta S, et al. End-to-end learning of geometry and context for deep stereo regression[C]//The IEEE International Conference on Computer Vision, 2017.
[13]	Wang Xin, Wang Xiangjun. Multiple targets sparse matching for binocular vision positioning system with large field of view[J]. Infrared and Laser Engineering, 2018, 47(7):0726001. (in Chinese)王鑫, 王向军. 大视场双目视觉定位系统中多目标稀疏匹配[J]. 红外与激光工程, 2018, 47(7):0726001.
[14]	Qiao Tieying, Cai Lihua, Li Ning, et al. Opposite target measurement based on infrared radiation characteristic system[J]. Chinese Optics, 2018, 11(5):804-811. (in Chinese)乔铁英, 蔡立华, 李宁, 等. 基于红外辐射特性系统实现对面目标测量[J]. 中国光学, 2018, 11(5):804-811.

Depth estimation technique of sequence image based on deep learning

doi: 10.3788/IRLA201948.S226002

Abstract

References

Proportional views

通讯作者: 陈斌, bchen63@163.com

Article Metrics

Related

Proportional views