The detection, identification and precise positioning of fruit under natural conditions based on machine vision is very important for the development of intelligent picking robots. In order to solve the problem that the traditional fruit detection model for the small target grapefruit missed detection and the leaf was falsely detected as the expansion period grapefruit, an improved feature fusion single multibox detector (InceptionV3-feature fusion single shot-multibox detector, IFSSD) was designed. The feature fusion single multibox detector (feature fusion single shot-multibox detector, FSSD) was used as a base detector and optimized in two ways. On the one hand, the improved InceptionV3 network was used instead of very deep convolutional networks 16 (VGG16) as the backbone network to improve computational efficiency. On the other hand, the Focal Loss function was used instead of the Multibox Loss function, which improved the mischeck ingresss of the detector due to the imbalance of positive and negative samples. Finally, the test data set was verified, and the results showed that the model achieved an average accuracy of 93.7% (mAP) (IoU was greater than 0.5). The time of one image was 29s. The model proposed can automatically detect the grapefruit in the grapefruit tree and locate it accurately in real time, which effectively promoted the development of intelligent picking robot.