In order to improve the efficiency and accuracy of cows body size measurement and reduce labor intensity, a cows body size measurement method based on key frame extraction and head and neck removal was proposed. Firstly, a cows top-down depth video acquisition platform was built, and the cows target in the depth image was extracted by using the watershed algorithm. Secondly, the contours of the left and right sides of the cows were obtained by using the image scanning strategy, and the key frames containing the complete cows torso in the image sequence were extracted by using the Hough transform-based line detection method. Then, according to the skeleton features of the cows-head area, itwas determined whether the head existed. If the head existed, the cows-head in the image was removed based on the convex hull analysis method, and the cows-neck was removed by the polynomial curve fitting method. Finally, according to the spatial characteristics of the cows-body measurement points, the straight length, shoulder width, belly width, hip width and body height of the cows were automatically calculated. The accuracy of the method was tested with 2.163 depth images of 35 cows. The analysis result showed that the accuracy of the key frame extraction method was 97.36%, which can effectively replace the manual selection of key frames. The accuracy rate of the head detection method was 94.04%, which improved the efficiency of the cows body measurement point positioning. The average relative error of body measurements was within 3.3%. The research results can effectively improve the efficiency and accuracy of automatic measurement of cows body size.