In order to improve the accuracy of muskmelon’s defect detection, an automatic defect detection system based on support vector machine (SVM) was set up by adopting complex features of texture and color. Four textural parameters and twelve color features of combinations from RGB were tested for the discriminability in stem, calyx, bruise and mildew. Through the experiments, two textural and four color features with good discriminability were selected and treated as the complex features. The results indicated that with the complex features and SVM, the accuracy of classification on muskmelons was up to 92.2%. 