Accurately estimating the severity of wheat leaf diseases can reduce planting costs and agricultural ecological environment pollution by the targeted application of pesticides, and contribute to the precise prevention and control of diseases in wheat field, while reducing the cost of the pesticide and the pollution of agricultural ecological environment. The stripe rust and powdery of wheat leaves were taken as the research object. For the problems of small differences in color and texture characteristics of images with different severities of the same diseases, and low classification accuracy of traditional methods, an improved convolutional neural network was proposed based on recurrent spatial transform (Recurrent spatial transformer convolutional neural network, RSTCNN), which was conductive for severity assessment of wheat leaf diseases. RSTCNN consisted of three scale networks which were connected by regional detection subnetworks. In each scale network, VGG19 was used as the basic classification subnetwork to extract the disease features. Furthermore, in order to fix the dimensions of the front and behind feature maps in the region detection process, the spatial pyramid pooling (SPP) was introduced before the fully connected layer. Spatial transform (ST) was used by region detection sub-network to effectively extract the attention region of the upper scale network feature map. The feature map of the disease image obtained by the convolutional pooling layer can be used as a key for predicting the category probability of the severities at multiple scales. Besides, ST were performed for the detection of region attention to serve as the input of the next scale network. The joint optimization and recursive learning of attention region detection and local fine-grained feature representation were carried out by means of alternating promotion. Finally, the output features of the different scale networks were merged, and then incorporated into the fully connected layer and the Softmax layer for classification, so as to realize the estimation of wheat leaf disease severity. The disease dataset of wheat leaf images with stripe rust and powdery mildew was built by data enhancement methods. Through experiments, results were found that the improved RSTCNN had a better accuracy in estimating the severity of network fused at three layer scales, reaching an accuracy of 95.8%. Compared with the basic classification network model, the accuracy rates were increased by 7~9 percentage points, which effectively enhanced the classification ability of the disease areas in the images. Compared with traditional machine learning algorithms based on color and texture features, the accuracy rates of RSTCNN were improved by 9~20 percentage points. The results showed that the proposed method significantly improved the estimation accuracy of wheat leaf disease severities.