In view of the lack of effective identification methods for grape cultivars identification under the field natural background, a residual network ResNet50-SE based on attention fusion mechanism was proposed to classify and identify grape varieties in different growth periods under natural background, and the identification effect of the network was analyzed and verified. The SE attention module was introduced into ResNet-50 network, and the recognition of grape shoots, young leaves and mature leaves in different periods was realized through transfer learning. Besides, in order to reveal the attention mechanism, the grape characteristics of different growth stages extracted from each layer of ResNet50-SE model were visualized and explained by the Grad-CAM visualization method. The t-SNE algorithm was applied to cluster the multi-features of different grape varieties extracted by the model, and then the performance of multi-features extraction of the model was intuitively evaluated. The results indicated that the ResNet50-SE network had a high recognition rate and strong robustness for grape multi-features recognition in different periods under the complex background conditions in the field. The accuracy rate of the model test set reached 88.75%, and the average recall rate reached 89.17%. Compared with AlexNet, GoogLeNet, ResNet-50 and VGG-16, the accuracy of the test set was improved by 13.61, 7.64, 0.70 and 6.53 percentage points. The attention mechanism can significantly reduce the influence of the background and strengthen the effective features. The model had a strong clustering effect on the features of different growth periods extracted from the training set. Therefore, the SE module can obviously improve the effect of ResNet-50 model in the feature extraction process, and effectively reduce the impact of field complex background on the classification results. The research result can provide a reference for the classification and recognition of grape cultivars multi-features under field complex background.