Parts of 20090716 and 20090730 HJ-1 CCD multi-spectral images were cut as the experimental areas. The thick cloud, thin cloud and cloud shadow were detected based on object-oriented method. And then, the substitution method was used to remove the cloud and cloud shadow. The comparison of pixel gray mean, maximum, standard deviation and entropy of original image and restored image indicated that the restored image accorded with the ground truth. The object-oriented and substitution could be an important method in the cloud detection and removal of HJ-1 CCD multi-spectral image.