To realize the automatic detection of pig ear-based surface temperature and reduce the error caused by fast discriminative scale space tracking (FDSST) in the head tracking in the thermal infrared video, an improved detection method of pig ear-based surface temperature by using the skeleton scanning strategy was proposed. Firstly, the initial frame of the video was preprocessed to extract the simplified pig skeleton. Secondly, the skeleton scanning strategy was designed to scan the key points of the head skeleton and realize the head positioning in the initial frame. Thirdly, FDSST was used to track the hog head. After each continuous tracking of N frames, skeleton scanning strategy was adopted to reposition the head and reduce the tracking frame drift. Finally, a method for extracting the surface temperature of the ear base was proposed. According to the temperature distribution of the left and right ear sides of the head, the temperature of the ear base was extracted and the error was corrected. The method was tested on the Matlab platform by using the collected videos of 30 pigs. Compared with FDSST, compressed sensing tracking and nuclear correlation filtering tracking, the average tracking accuracy of the proposed method was improved by 7.82 percentage points, 11.82 percentage points and 8.78 percentage points, respectively. The maximum error of the extracted ear base surface temperature was 0.32℃. This study can provide technical support for automatic detection of pig ear base surface temperature.