Abstract: An optimized algorithm of automatic panoramic image mosaic was presented based on time and frequency domains. Firstly, the improved phase correlation method was used to sort the panoramic image sequence automatically and then determine the overlap region. In order to reduce the false match rate of the corner, the brightness of the overall image was adjusted automatically according to the overlap region's pixel mean. Then, an improved Harris operator was adopted to extract corners (with no need to set threshold value manually), the initial feature points pairs were obtained by the bidirectional greatest correlative coefficient and the false feature point pairs were rejected by RANSAC algorithm. Finally, the nonlinear smoothing algorithm was adopted to fuse the overlap regions of the image. The experimental results show that optimization algorithm is simple and effective in the sorting process. Compared with the existing algorithm, the optimization algorithm proposed at present makes a great progress in the success rate and the efficiency in the matching and extracting process of feature points. The stitched image is with great clarity, soundness and stitching accuracy.