A method for fruit recognition and feature extraction based on the color and shape features of kiwifruit in nature was studied. It could reduce the influences of complicated background, different kiwi growth state and natural lighting condition. First, R—G color component was chosen by comparing different color spaces. Then the optimum partition coefficient of nR—G color characteristics was determined according to the image evaluation method of error segmentation pixel, and 0.9R—G was selected finally. The Otsu method was used for threshold segmentation and morphological operation was employed to remove residual noise, and then the regions of target fruits and backgrounds were successfully separated. The image boundary was extracted by Canny operator and consequent elliptic Hough transform, which made the target fruit be recognized separately. Also, fruit features as centroid coordinates, long axis end coordinates, long axis length and short axis length were extracted. By using this method, 49 images including 110 fruits were tested. Test results demonstrated that the recognition ratio of separate fruit, adjacency fruit, partial sheltering fruit and overlapped fruit were 96.9%, 92.0%, 86.6% and 81.6%, respectively.