Abstract: The infrared target under complex ground scene is vulnerable to the background and is often occluded. It is difficult to detect and track targets simply rely on brightness or gradient information. According to the difference of velocity field between the moving target and the complex ground, a new target tracking algorithm with optical flow was proposed. Firstly input images were aligned to ensure a relatively static background in the servo track; secondly the improved Horn-Schunck optical flow was calculated in the tracking gate; lastly the transition probability of the particle in the particle filter algorithm was optimized with optical flow features of the target. Experimental results show that the proposed tracking algorithm can keep track of the infrared moving target under complex ground scene, and is not subject to the impact of short-time occlusion.