In order to accurately segment diseases and insect pests on crop leaves, a Gaussian mixture model (GMM) based texture difference expression method and an accurate segmentation approach based on improved distance regularized level set evolution (DRLSE) were proposed. Considering that crop leaf generally has a certain texture feature, GMM was used to characterize the leaves texture feature, and improved DRLSE which took full advantage of texture difference information between objects and background, was employed to get the accurate contour of diseases and insect pests. Firstly, the texture feature of leaf sampling area was modeled by using GMM. Secondly, the texture difference between the pixels in the diseases and insect pests’ area and the sampling area was calculated, and the texture difference image was obtained at the same time. Thirdly, the initial contour for DRLSE was obtained by Otsu and morphologic post processing. Finally, the contour of diseases and insect pests was evolved accurately with texture difference guided DRLSE. The experimental results show that the diseases and pest contour can be obtained accurately with the introduced method and also can provide the basis for the subsequent identification and prevention of crop diseases and pests.