In order to achieve accurate, non-destructive and rapid classification of tea leaf species, the tea leaf species classification was realized through convolutional neural network by taking the images of tea leaves of six varieties under complex background as the research object. The classic lightweight convolutional neural network SqueezeNet was selected, and by adding batch normalization processing in the Fire module, the network parameters were not significantly increased to greatly improve the accuracy of the classification of multi-variety tea leaves. The 3×3 standard convolution kernel was replaced with a depthwise separable convolution, which further reduced the network model and reduced the networks requirements for hardware resources; by introducing an attention mechanism into each Fire module, the networks extraction of important feature information was enhanced. The test results showed that the original SqueezeNet model had an accuracy rate of 82.8% for the classification of multi-variety tea leaves, and the model after adding batch normalization had an accuracy rate of 86.0% in the test set, and the number of parameters was only 7.31×105, compared with the parameters before improvement. The amount of calculation was only increased by 0.8%, and the amount of calculation was basically the same as that before the improvement; the model after replacing the 3×3 standard convolution kernel in the Fire module with a depthwise separable convolution model had an accuracy rate of 86.8% in the test set, and the accuracy rate was increased by 0.8 percentage points, the amount of parameters were decreased to 2.46×105, the model parameters were decreased by 66.3%, and the amount of computation was decreased by 60.4%; the classification accuracy of the model test set after the introduction of the attention mechanism reached 90.5%, which was increased by 3.7 percentage points, while the amount of parameters was only increased by 1.23×105, and the amount of operations was only increased by 2×106. The improved model was further compared with the classic models AlexNet, ResNet18 and the lightweight networks MobilenetV3_Small and ShuffleNetv2. The results showed that the improved model had the best comprehensive performance in the classification of multi-variety tea leaves, and the three indicators of model scale, classification accuracy and classification speed were well balanced.