Based on the method of adaptive global threshold and markers fusion, an algorithm was proposed in order to solve the problems of over-segmentation and the under-segmentation caused by incomplete marking, which might occur concurrently during the segmentation of remote sensing building images. First, the algorithm was used in wavelet transform to generate image gradient according to the distribution and texture characteristics of buildings, and the generated gradient image was filtered through morphological reconstruction. Then, the background markers were extracted by local minimum and the building makers by adaptive global threshold. After both markers were fused, they were used to modify the weighted pixel Sobel gradient image for accurate image segmentation. The experimental results demonstrated that the algorithm could make up for a lack of the local extreme marker, and significantly inhibited both over-segmentation and the under-segmentation. As a result, the segmentation accuracy reached to 90.7%.