Abstract: Automatic marking points extracting with high accuracy is basic for calibration, stereo calculation, visual navigation and other applications. The bottle-neck is the accuracy and robustness. Based on the rough point from the cross of lines of the neighbor barycenters in two directions, the ridge points were detected by the symmetric function and refined by an iterative least square method, based on the refined ridge points, the Hough transform was improved for determining the lines and their intersections. At last, the position of the marking points was to be adjusted by the penalty function with a changing step along the descent direction both in deflection and rotation separately. Our algorithm is robust to noisy, rotation, illumination variations and a certain degree of distortions, the accuracy could be less than 5 percents of a pixel.