Aiming at the problem that the basic convolutional neural network is vulnerable to background interference and the expression ability of important features is not strong in apple orchard pest recognition, an apple orchard pest recognition method based on improved Mask R-CNN was proposed. Firstly, based on Haar feature method, the apple orchard pest images collected from multiple points were iteratively preliminarily segmented, the single pest image sample was extracted, and multichannel amplification on the sample was performed to obtain the amplified sample data for deep learning. Secondly, the feature extraction network in Mask R-CNN was optimized, and the ResNeXt network embedded in the attention mechanism module CBAM was used as the Backbone of the improved model, which increased the extraction of pest space and semantic information by the model, and effectively avoided the influence of background on performance of the model. At the same time, the Boundary loss function was introduced to avoid the problem of missing edge of pest mask and inaccurate positioning. Finally, the original Mask R-CNN model was used as the control model, and the mean average precision (mAP) was used as the evaluation index to conduct experiments. The results showed that the mean average precision of the improved Mask R-CNN model reached 96.52%. Compared with the original Mask R-CNN model, the mean average precision was increased by 4.21 percentage points. The results showed that the improved Mask R-CNN can accurately and effectively identify pests in apple orchards. The research result can provide technical support for green control of apple orchard pests and diseases.