The critical task of the robot vision system in the apple harvesting robot is to recognize and locate each single apple. To solve recognition problems such as the big error, too much calculation and time consuming, a new recognizing method, support vector machine (SVM) was applied to improve recognition accuracy and efficiency. Firstly, vector median filter method was used to remove the color images noise of apple fruit. Secondly, segmentation of the images based on region growing method and color properties was done. At last, color properties and shape properties of color image were extracted, and the classification method of SVM for recognition of apple fruit was used. Experimental results indicate that the classification performance of SVM is better than that of neural networks. Recognition rate of apple fruit based on SVM combined with color and shape properties is higher than only using the color or shape properties. 