In order to realize accurate classification of tea diseases, a disease image classification method based on SimAM-ConvNeXt-FL model of migration learning was proposed to address the small sample problem and uneven distribution of categories in tea disease image classification. Firstly, an SimAM module was added to the ConvNeXt model to enhance the extraction of complex features. Secondly, to address the problem of uneven sample distribution, the Focal Loss function was used as the loss function in the training process, and the effect of uneven sample distribution was reduced by increasing the weights of a smaller number of samples. Finally, the SimAM-ConvNeXt-FL model was used to train the Plant Village dataset, and the parameters obtained from the training were migrated to the measured tea leaf disease images and fine-tuned to reduce the impact of overfitting, and ablation experiments were set up to prove the validity of the model improvement, and comparison experiments were carried out with the different classification models AlexNet, VGG16, and ResNet34 models comparison experiments were conducted respectively. The experimental results showed that the SimAM-ConvNeXt-FL model had the best recognition effect, with an accuracy of 9648%, and the F1 values of the SimAM-ConvNeXt-FL model compared with the original ConvNeXt model for tea coal disease, tea phoma, tea anthracnose, healthy leaves, and tea white star disease were improved by 4.46 percentage points, 3.76 percentage points, 0.43 percentage points, 0.22 percentage points, and 5.23 percentage points respectively. The results showed that the model proposed had high classification accuracy and strong generalizability, which can promote the development of tea disease classification.