In view of the inefficiency and lag of the traditional artificial diagnosis and classification methods for grape downy mildew, an improved residual network model for grape downy mildew identification and disease degree classification was proposed. The images of downy mildew in the prophase, metaphase, anaphase and healthy leaves were collected in the field, and the effects of the factors of weather, shooting angle and equipment noise were simulated to increase the data capacity. Based on the characteristics of high similarity and difficult to distinguish between leaves with different disease degrees, by using the optimized ResNet-50 model, a 3×3 maximum pool layer with step size of 2 was added into the Base Block of Conv3, Conv4 and Conv5 (the residual body composed of several residual blocks) to solve the problem of serious information loss of the shortcut branch and the insufficient feature extraction ability of the main branch, so as to achieve dimensionality reduction of retaining important information. The main branch structure of the residual block in the ID Block was improved, and the 1×1 dimensionality reduction convolution layer in the first layer was replaced by 3×3 dimensionality reduction convolution layer with a step of 1;a newly full connection layer was designed, in which the global average pooling and 3 layer full connection layer network were used to replace the original model full connection layer structure, and the Dropout (random inactivation) layer was added to avoid the model over fitting. The experimental results of the original data set and the expanded data set showed that when the momentum factor m was 0.60 and the learning rate α was 0.001, the improved ResNet-50 network model had the best recognition effect compared with ResNet-34/50/101, AlexNet, VGG-16 and GoogLeNet. The improved residual block enhanced the feature extraction ability of the network. On the basis of optimizing the super parameters, the accuracy of the improved residual block was 2.31 percentage points higher than that of the original model. Different data augmentation methods had certain contribution to improve the recognition accuracy of the model. The recognition accuracy of the improved residual network model was 4.68 percentage points higher than that of the original model, reached 99.92%, which provided a real-time and accurate solution for automatic classification of grape downy mildew disease degree in complex environment.