The automatic picking system of fruit harvesting robot was designed based on stereo vision. Firstly, the fruit objects were recognized in image space by using the Hough transform. Some pixels of each object obeying the uniform distribution in the image space were sampled randomly. Then the sphere model of fruit was constructed in 3-D space with these pixels, and the coordinates of the center of each fruit object were obtained directly. Secondly, hand-eye calibration was carried out through the least square method. Finally, the path planning of the harvesting robot was analyzed simply. Experimental results showed that the effects of the partial occlusion and invalidation of stereo matching in the presence of complex scene on the automatic picking system was weakened, and the positioning accuracy of the fruit object was less than 8mm. Therefore the reliability and precision of fruit gripping were improved significantly. 