System design of multi-resolution microscopic correlation imaging based on deep learning

  Objective   Microscopic imaging technology is the primary research method for biological organs, tissues and cells. It plays a significant role in promoting the development of biology and medicine. However, the diversity and complexity of biological samples, the low signal-to-noise ratio, and the optical diffraction limit of traditional optical microscopy significantly limit its application. Different biological samples and different application scenarios have different requirements for microscopic imaging technology. Therefore, in clinical applications, how to obtain images with appropriate resolution through practical needs, and how to shorten imaging time while ensuring imaging quality are the problems that need to be solved urgently in microscopic imaging applications.

  Methods   The microscope is modified by adding a beam-splitting device in the optical path. The light that carries the sample information was exported to the multi-resolution microscopic correlation imaging system after being magnified by the objective lens. The experimental system was integrated into the shell (24 cm×18 cm×12 cm, Fig.2). The optical signal is illuminated to DMD, and the signal light is modulated by DMD and received by a single-pixel detector. The reconstructed images of the sample are obtained through the second-order correlation operation of the modulation matrix and detection intensity of a single-pixel detector. The imaging system was equipped with an industrial computer and a data acquisition card, which are used to control the DMD, load the preset pattern and record the light intensity collected by the detector. The reconstructed images of the sample are obtained through the second-order correlation operation of the modulation matrix and detection intensity of a single-pixel detector. Then the images are processed through deep learning.

  Results and Discussions   The tissue slice was used as the target object, and the performance of the DLGI system after hardware and software design were tested. The imaging results under five different sampling rates were obtained (image resolution: 128×128, Fig.5 and Fig.6). With the decrease of the sampling rate, the imaging quality is reduced significantly, accompanied by a large amount of noise. When the sampling rate reaches 60%, the internal details of biological tissue in traditional correlation imaging (GI) cannot be recognized, and it is unacceptable for pathological section observation. The image quality is significantly improved after using the deep learning method. Even when the sampling rate is 60%, the internal details and edge contours of biological tissues can be restored clearly, and the image noise is significantly improved. In this paper, the ultra-efficient and lightweight hyper-division network based on heavy parameterization reduced the complexity of image calculation significantly (Fig.3), and the reasoning time can reach 51 ms. The imaging time of the imaging system in this paper can save 0.37 s while ensuring the imaging quality and significantly reducing the memory occupation (Tab.1) .

  Conclusions   A multi-resolution microscopic correlated imaging based on deep learning is designed to meet the diverse needs of microscopic imaging and solve the contradiction between imaging quality and imaging time in practical application. The system combines deep learning with correlation imaging technology through hardware design and software processing of the microscope. The imaging system can restore image details with a sampling rate of only 60%, significantly reduce the noise caused by under-sampling, and significantly improve the time resolution of the system. In addition, to meet the actual needs of the small-scale imaging systems, an ultra-efficient super-resolution network is adopted based on the overparameterization method to realize real-time imaging under equipment with limited resources. The proposed imaging system can significantly reduce the imaging time and memory occupation while maintaining imaging quality. The test results of different types of biological samples and resolution boards further show the robustness and anti-noise performance of the system. The research results of the system have great significance for the biomedical field.