Abstract: For the rotation, translation, scale invariant properties of SIFT(Scale Invariant Feature Transform) feature, it has been widely applied in imaging matching. But there are two defects of using SIFT while matching. Firstly, the matching performance is directly affected by the matching parameters, and there is always mismatching and error matching existed. Secondly, it only fits for matching under similarity transformation, while at the affine transformation situation it fails. In this paper, a novel iterative matching algorithm based on transformation estimation was proposed. The SIFT matching problem was turned into an optimization problem about SIFT feature vector and the geometry distribution of the point sets. By searching for the affine transformation and correspondences under the iterative deterministic annealing frame, the algorithm got the optimal matching result of SIFT point sets. Experiment results show that even at large affine transformation, the algorithm can still get the right matching results.