Abstract: The picture quality of the video is very poor under the low illumination night environment, it often appears low contrast, blurred, color offset, etc, and it has influence on interpretation and understanding of the video. For solving the problem of nighttime video, the different time space background fusion technology was proposed based on moving object and a new Retinex image enhancement algorithm. This method improved the whole brightness and the contrast of the video through the strategy of adaptive brightness adjustment. Then the illumination components of brightness images were extracted by the trilateral filter based on the theory of Retinex. The reflecting components of images were obtained through compressing illumination. It was because that the reflecting components of images included large number of particulars and marginal information of images, reflecting components were enhanced through integrating Sigmiod non-linear stretching function. At last, it enhanced the saturation component to get more bright colors. The experiments show that the algorithm can improve the visual effect of the low illumination video at nighttime. The image brightness, contrast, sharpness is improved, and the method not only enhanced the video image detail, but also retained the important content of the video. The method avoid the distortion of images, such as, halo, ghost, color deviation.