In order to solve the problem of low picking efficiency of harvesting robot under the target fruit oscillation condition, a recognition and location method of oscillating fruit based on monocular vision and ultrasonic testing was proposed. Firstly, the acquired sequential images of oscillating fruit were segmented by using Otsu algorithm based on R-G color component; morphological operation was employed to eliminate residual noise; and the region of target fruit was separated from backgrounds. Secondly, the region of target fruit was filled with gray threshold. Thirdly, the processed sequential images were superimposed and a composite image was obtained, and the 2-D centroid coordinates of the oscillating fruit were extracted from the fruit oscillating region in the composite image. Fourthly, the picking manipulator was controlled to move and its end was pointed to 2-D centroid coordinates, at the same time, the depth information of the target fruit was acquired through ultrasonic testing, and the peak-to-peak value of ultrasonic echo signal was extracted for recognition of fruit and tree. Finally, the end-effector of picking manipulator was started to grab the fruit when it was within valid range and the manipulator was stopped motion. The experiments results showed that the success rate of picking can reach 86%. This method is universal and suitable for apple, citrus, pears and other spherical fruits. The research can provide a reference for realization of harvesting robot practicability.