A novel algorithm based on nonsubsampled Shearlet transform (NSST), Gaussian multi-scale space and anisotropic diffusion was proposed for detecting crack defects with uneven background, low contrast, noise corruption and textured interference in magnetic tile surface images. Firstly, NSST was employed to decompose the source magnetic tile image into one low-pass subband and a series of high-pass subbands. Then the anisotropic diffusion and the modified γ enhancement method were applied to remove the noise and enhance the weak object information in the high-pass subbands, respectively. Meanwhile, the background was estimated in the Gaussian multi-scale space constructed by convolving the low-pass subband with a varied two-dimensional Gaussian functions, and the even low-pass object could be obtained by using background subtraction. Finally, inverse NSST was utilized to reconstruct the enhanced object image which was free from noise and grinding texture interference, and crack defects could be segmented from the reconstructed image by applying the adaptive threshold method and regional connectivity function. Experimental results demonstrate that compared with four existing methods (OTSU method, method based on the adaptive morphological filtering, method based on Curvelet transform and texture feature measurement and method based on Shearlet transform), the proposed method achieves better performance in terms of defect detection accuracy.