A kind of dynamic recognition method under the fruit oscillation condition for the harvesting robot was researched in order to resolve the problem of fruit oscillation influence to its recognition and orientation, and further influence to the harvest efficiency of harvesting robot. Firstly, the acquired oscillation fruit was segmented into two parts: fruit image and background image; secondly, the inter-frame difference method and the minimum enclosing rectangle method in horizontal direction were introduced to label dynamic region of the oscillation fruit for the segmented image; thirdly, the oscillation fruit was identified and located. The harvesting oscillation fruit was selected based on the principle of the nearest to the image center when the image had a lot of oscillation fruits. The test result showed that the oscillating fruit was nicely recognized and the recognition time was less than 0.5s for the most cases under the actual harvesting environment.