Abstract: The operational principle and characteristic of incremental photoelectric angular encoder were summarized and discussed. And the reasons of reference mark excursion for incremental photoelectric angular encoder were analyzed. In order to validate reference mark excursion, a pointing function, which defined in the spherical coordinate of level mounting telescope, was expanded into series by spherical function. Then the spherical function model of pointing error was obtained. 30 stars scattered in sky space uniformly were selected to observe, through which the discrete value of 30 stars' pointing error were gained in longitude dimension and latitude dimension respectively. A new technique called external checking method, which used observing data two days ago to check intraday pointing error, was tested to validate the phenomenon of reference mark excursion. An experiment measuring pointing error before and after reference mark calibration by observing Polaris last 11 days. The results showed reference mark excursion caused pointing error of telescope becoming greater than the technical index designed. And after calibration the reference mark became fixed all night. To evaluate the effect of reference mark calibration, orbital position of a GPS satellite was measured by optoelectronic telescope. Pointing error of the telescope was gained compared with precision orbital position of GPS satellite. The results show that before reference mark calibration the mean azimuth and elevation error was 13.99, 11.50. And after calibration the mean azimuth and elevation error decreased to 5.94, -3.49 respectively. The experiment of GPS star observation showed that the reference mark calibration was available to eliminate reference mark excursion and to improve observation precision.