There is a high application demand for accurate counting of disordered targets in agricultural environments, and such counting plays an important guiding role in their biomass and biological density management. In the process of larvae of black soldier fly target tracking, the tracking object has the characteristics of high speed and non-linearity, and the conventional algorithm has the problems of insufficient speed of tracking target and difficulty of re-identification after losing the target. To address these problems, an improved SORT algorithm was proposed, which improved the speed and accuracy of the target tracking algorithm by improving the Kalman filter model, and enhanced the counting accuracy. In addition, for the complex background problem caused by larval trait diversity and mixing in the process of black gadfly larval target recognition, the target recognition accuracy was improved by experimentally comparing the performance of multiple deep learning networks, which selected YOLO v5s algorithm to extract multidimensional features of images. The experimental results showed that in terms of delineation counting, the improved SORT algorithm improved the average accuracy by 4.19 percentage points compared with the original model, from 91.36% to 95.55%, and the effectiveness of the model was proved through simulation and practical application. In terms of target recognition, using the YOLO v5s model on the training set achieved a frame rate of 156f/s, mAP@0.5 value of 99.10%, accuracy of 90.11%, and recall rate of 99.22%. Its overall performance was better than other networks.