Artificial identification of wheat rust is costly and inefficient, and can no longer meet the needs of modern agricultural production. A lightweight dense multi-scale attention network model called Mobile-DMSANet was presented for the automatic identification of rust on wheat leaves (stripe rust and leaf rust) from images of natural scenes taken in the field. In the input layer of the network, a fast subsampling block (FSB) was used to improve the feature expression ability of the network without adding computational cost. In the feature extraction layer, three lightweight blocks called dense multi-scale attention (DMSA) blocks were used to extract the features of rust on wheat leaves. In the DMSA block, a multi-scale three-way convolution (MSTC) layer was designed to get different scales for the receptive fields, in order to improve the expressive ability of the network and its ability to perceive the features of rust disease at different scales. Six MSTC layers were used to achieve feature reuse by dense connections in the DMSA block, an approach that not only greatly reduced the number of parameters of the network but also improved the feature extraction ability for similar diseases. A coordinated attention (CA) block was also introduced to the DMSA block to increase the sensitivity to positional information and suppress background information in the image. The output layer of the network used a Softmax function to classify rust on wheat leaves. The results showed that the recognition accuracy of Mobile-DMSANet model on the test dataset was 96.4%, which was higher than that of other models. Mobile-DMSANet had only 454000 parameters, less than for other lightweight models. The proposed model can be used for the automatic identification of rust on wheat leaves using mobile devices.