Aiming at the problem of jujube variety identification in natural scenes, machine vision technology with jujube fruit as the research object has become one of the mainstream methods for accurate identification of jujube varieties. However, due to the small inter-class difference and large intra-class difference of jujube varieties, it is difficult for a single organ to fully express the different characteristics of jujube varieties. A method of jujube varieties recognition based on multi-organ feature fusion was proposed. Firstly, the YOLO v3 detection algorithm was used to segment and extract the jujube fruit and leaf organs in the collected natural scene images, and a multi-sample dataset of jujube varieties based on Cartesian product was proposed to construct two organ combination pairs, and then based on the EfficientNetV2 network model, a fusion strategy that can fully learn the correlation between the characteristics of the two organs was designed to improve the model performance, and a stepwise transfer training method was introduced to improve the recognition efficiency of jujube varieties. Finally, a large number of experiments were carried out on the constructed dataset containing 20 jujube varieties, and the recognition accuracy of 97.04% was obtained, which was significantly better than that of the existing research results, and the training time and convergence speed of the proposed method were also improved. The results showed that this method can effectively integrate the characteristic information of jujube fruit and leaf organs of jujube cultivars, which can provide valuable reference for other variety identification research.