Abstract: In the process of target tracking with complex background, changes in the target brightness and contrast easily lead to instability or failure of target tracking. To deal with such problem, a new correlation tracking approach was presented which was robust to the variation of brightness. Taking advantage of the merit of phase congruency that was insensitive to the variation of image brightness, the phase congruency was calculated firstly from the original image, and then the traditional MAD (Minimum Absolute Difference) correlation tracking approach was used to track the target. Experimental results show that the proposed algorithm is considerably adaptive to the variation of image brightness. It can successfully solve the problem of the tracking point excursion and even the failure of tracking which the traditional tracking methods is unable to deal with when the image brightness changes.