Plant pest and disease is one of the three major natural disasters. Pest identification tends to consume a lot of labor, and it is difficult for naked eyes to quickly and accurately identify pest species. However, there still exist some drawbacks in the traditional deep learning algorithms for pest recognition, such as gradient explosion or gradient disappearance in deep neural networks, degradation and overfitting caused by limited sample size. In order to address these problems and improve the accuracy of pest recognition, a pest image recognition method based on improved residual network was proposed. Firstly, the pest images in the data set were converted to grayscale before edge detection was performed on them by using Richedge. To obtain a finelined pest image, the Richedge was combined with median filtering, Sobel operator and Canny operator to detect the edges of the pest images. Among them, the median filter effectively eliminated the salt and pepper noise, the Sobel operator accurately detected the position information, and the Canny operator detected the weak edge. The images after edge detection were quantized to be 224pixel×224pixel for training and classification. Then the obtained pest image set was used to train the deep neural network, which was a variant of standard residual network with additional convolution layers and channels for extracting more image features. And the dropout layer was added to each residual block of the network to prevent overfitting when it was trained on a relatively small data set. Besides, the regularization hyper parameters of the network were designed to be optimized by Bayesian method which adaptively adjusted the size of the hyper parameters with the adjustment of weights during network training. The weights of the proposed network were optimized through the Blockcg algorithm. In the optimization algorithm, the block diagonal was used to approximate the curvature matrix, which improved the convergence of the Hessian matrix; and independent conjugate gradient update was conducted for each subblock, which divided the whole issue into certain number of subproblems and reduced the complexity of local search. Eventually the values of the weights were not updated until an ideal pest classification accuracy rate was obtained. To verify the validity and robustness of the proposed method, an image data set of 38 common garden pests in north of China was collected and experiments were carried out on this data set. Experimental results empirically demonstrated that compared with the three traditional pest recognition methods for the same data set, the proposed method could make the recognition accuracy increase by 9.6 percentage points on average and the weighted average score increase by 16.3 percentage points, 10.8 percentage points and 4.5 percentage points, respectively.