The identification of plant species plays an important role in the research and development of botany. In order to improve the recognition accuracy of plant leaf images, taking into account the plant leaf database as a small sample database, a convolutional neural network plant leaf image recognition method based on transfer learning was proposed. Firstly, the plant leaf image was preprocessed, and the plant leaf image data set was expanded by random horizontal, vertical flipping, and random scaling of the original image. The expanded plant leaf image data set samples were de-averaged and divided into 4∶1 proportions as training set and test set. Then, the pre-trained models (AlexNet and InceptionV3) on the large-scale datasets were loaded to the plant leaf image data set. The parameters of all convolution layers of the pre-training models were retained, but the last layer was completely replaced to adapt the task of plant leaf image recognition. Finally, the proposed methods were compared with the support vector machine (SVM) method, the deep belief nets (DBN) method and the convolutional neural network (CNN) method from training time and the accuracy of test set in the ICL database. Tensorflow was used to train the network model in the experiment, and the model was visualized by the Tensorboard. The results showed that the accuracy rate of test set obtained with AlexNet and InceptionV3 pre-training models were 95.31% and 95.40%, respectively, which were better than those with SVM method, DBN method and CNN method. The training time of AlexNet and InceptionV3 pre-training model were 26220s and 3984s, respectively. The InceptionV3 pre-training model was superior to the AlexNet pre-training model in terms of test set accuracy and training time.