The picking robot in unstructured environment has practical value in visual localization of disturbed grape. Firstly, the shape of grape was analyzed based on model of “flexible rod-hinge-rigid rod-mass ball” and its disturbed state was decomposed into pendulum of XOY plane and YOZ plane, and then fruit and stalk of grape were obtained by Otsu function from multi frame image in video of which captured disturbed grape. Then centroid of grape in each frame image was calculated, and curve fitting of grape centroid of each frame image was made. The period of grape pendulum movement and swing angle were calculated to judge whether current disturbed of the grape was suitable for visual positioning. For disturbed grape that can achieve visual positioning, choosing grape images corresponding to location in the middle of centroid point of pendulum motion, Canny edge detection was carried out on the upper rectangular region. The method of Hough line fitting and angle constraint were used to localize picking point of disturbed grape. Experiment results of visual localization showed that the accuracy of visual localization for disturbed grape picking points under different illuminations were more than 80% in natural environment, which provided theoretical basis for the application of picking robot in practical production.