Single-photon LiDAR imaging method based on sensor fusion network

LiDAR systems with active illumination obtain depth information of the scene using Single-Photon Avalanche Diode(SPAD) detectors to record the arrival time of reflected photons from the laser pulse. However, there is ambient light that interferes measurements during the detection period. Sensor fusion is one of the effective methods for single-photon imaging. Recently, many data-driven methods based on intensity-LiDAR fusion have achieved gratifying results, but most of them use the scanning LiDAR which has a slow depth acquisition speed. The advent of the SPAD array can overcome the limitation of frame rates. The SPAD array allows the collection of multiple returned photons at the same time, which accelerates the information collection process. However, the spatial resolution of SPAD array detectors is typically low, and the detection process is also interfered by the ambient light. Therefore, it is necessary to break the inherent limitation of the SPAD array through an algorithm to separate the depth information from the noise. In this paper, for the SPAD array detector with the array size of 32×32 pixel, a convolutional neural network was proposed, which could reconstruct high-resolution clean TCSPC histogram under the guidance of the intensity image. A multi-scale approach was adopted to extract input features, and the fusion of depth data and intensity data was further processed based on the attention mechanism in the network. In addition, a loss function combination suitable for the TCSPC histogram data processing network was designed, where the overall distribution of photons and the ordinal relationship between time bins in the temporal dimension could be simultaneously considered. The method proposed in this paper can successfully increase the depth spatial resolution by 4 times, and the efficacy of proposed method is verified on realistic data, which is superior to state-of-the-art methods qualitatively and quantitatively.