Abstract: Towards high resolution remote sensing images, combining with features of buildings, a novel method to extract buildings based on multi-layer level set framework was proposed. Firstly, as far as the impact of shadow and vegetation was concerned, it should be removed on the basis of the separation of gray value thresh and the joint distribution of hue and saturation. Then, an improved C-V level set segmentation algorithm combining with building features of roof's gray and obvious boundaries was applied to extract building regions of similar gray-scales on each gray layer, and thus all building regions of different gray-scales could be extracted layer by layer, followed by layers of segmented regions integration. Finally, the non-building regions were excluded by using normal areas of buildings and related position between buildings and shadows. The experiment results demonstrate that, compared with the traditional level set methods, this one can detect each single building of gray heterogeneity and buildings of multiple shapes and different gray-scales. Meanwhile, compared to the traditional C-V method, it largely reduces the leakage segmentation ratio by 25% and over-segmentation by 22%.