Accurate and effective identification of fish images play important role in the observation of fish populations and the management of the ecological environment. However, there were some issues, such as lots of kinds of fish, difficulty of data collection in the complex environments, and fine-grained fish image recognition. For solving the problem of few image annotation of fish image, a few-shot learning method based on metric learning was proposed. Firstly, the residual block structure of ResNet18 was used to improve the few-shot learning network based on metric learning, for extracting the deep features of fish images, and then they were mapped to the embedding space for obtaining the mean center by clustering skills. Secondly, for further improving the recognition accuracy, the improved few-shot learning model was used for pretraining on the mini-ImageNet dataset, and then the training results were transferred to the Fish100 fine-grained dataset for fine-grained training to get the final discrimination model. Based on this model, comparative experiments were conducted with the existing five few-shot learning models on the fish data set Fish100 and ImageNet. The results showed that the model proposed had the best recognition effect and the recognition accuracy on the two datasets reached 94.77% and 91.03%, respectively, and the accuracy, recall rate, and F1 were significantly better than that of other models. The experiments showed that the method proposed can effectively improve the accuracy of few-shot learning in fish identification with few annotated fish images, which can provide technical support and reference for the application of practical fish image recognition.