To improve the applicability and efficiency of scaffolding cultivation kiwifruit harvesting robot in orchard, an efficient and accurate recognition method for multiple clusters characteristics of kiwifruits under farview and occlusion environment conditions were researched. Faster R-CNN was proposed for detecting kiwifruit. The Im-AlexNet model was used to recognize the farview and occluded fruit image, including the sunny backlight, sunny rembrandt light, cloudy, night with illumination condition. In addition, there was more obstructive among fruit clusters and branches and leaves of fruit trees. By modifying the number of nodes in full connection layer of AlexNet model by transfer learning, finetuning the number of nodes full connection layer L6, L7 to 768 and 256. The feature extraction of kiwifruit was more accurate, and the recognition result of occluded image of fruit contour was obtained. Through the recognition of 1823 multicluster kiwifruit images trained by Im-AlexNet, the experimental results indicated that the average precision (AP) of farview and occluded complex condition images was 96.00%, and the recognition speed reached 1s. By comparing with LeNet、AlexNet、VGG16 models of training the same datasets, the AP of Im-AlexNet was 5.74 percentage points higher than those of Faster R-CNN network, and the rate of false recognition and missing recognition of kiwifruit was reduced by Im-AlexNet. It was proved that deep learning can solve the problem of recognition results of farview complex weather and occluded fruit, and kiwifruit harvesting robot was suitable for kiwifruit detection in complex environment, it can also be applied to other farview multitarget and partially occluded target recognition.