In order to judge fruit quality real timely, three conditions of litchi fruit, immature, mature and appearance rot after mature, were analyzed by using the fruit images of different growth periods in natural environment. The YCbCr color space model was selected and the exploratory analysis method was used to analyze and estimate the Cr component of litchi images of different parts, different illuminations and different growing periods, and the threshold ranges of Cr components of mature and immature litchi fruits was determined; for mature litchi, the fruit edge detection and Hough circle fitting processing were carried out on the Cr component diagram to mark litchi fruits. And then the texture statistics and the method that combining the color feature and area ratio of different parts of litchi fruit were used to judge litchi fruit deterioration. Finally, the vision intelligent judgment for the immature, mature and appearance rot of litchi fruit was realized and an intelligent system to identify litchi fruit quality was built. The test results show that the accuracy of identify litchi fruit quality is 93%.