Using computer vision technology to accurately and quickly identify the neck, ribs and spine of the sheep skeleton is the key to the development of a vision system for the slaughter and segmentation robot. To this end, the sheep skeleton image generation technology and the ICNet-based real-time semantic segmentation method of the sheep skeleton image were studied. DCGAN, SinGAN and BigGAN, three kinds of generation confrontation network were used to generate image effect comparison, BigGAN was selected to generate sheep skeleton image and expand the amount of sheep skeleton image data. On this basis, the generated image and the original image were combined to establish a combined data set, and then transfer learning was introduced to train ICNet and save the optimal model to obtain the segmentation accuracy of the model for the three parts of the sheep skeleton, MIoU and the average processing time of a single image, and it was taken as the criterion for the effect of semantic segmentation of sheep skeleton images, in the end, the optimal model’s segmentation accuracy and MIoU for the three parts of the sheep skeleton were 93.68%, 96.37%, 89.77%, and 85.85%, 90.64%, 75.77%, and the average processing time for a single image was 87ms. Then, the sheep skeleton images under different lighting conditions were simulated to judge the generalization ability of ICNet. Finally, comparing with the commonly used U-Net, DeepLabV3, PSPNet, Fast-SCNN four image semantic segmentation models to verify ICNet’s comprehensive segmentation ability, and the optimal weight was found by comparing the network segmentation accuracy under different weights of the middle resolution branch. The test results showed that the segmentation accuracy and MIoU of ICNet and the first three models were not much different, but the processing time was reduced by 72.98%, 40.82% and 88.86%. In addition, Fast-SCNN single image processing time was 43.68% higher than that of ICNet, but the MIoU was reduced by 4.5 percentage points and the resolution branch weight was 0.42, ICNet segmentation accuracy reached the highest. Combined with the experimental results, it was showed that this method had high segmentation accuracy and good real-time performance, the comprehensive segmentation ability was the best, and it had a certain generalization ability. The above research provided theoretical support and technical support for the research and development of the intelligent segmentation robot vision system for sheep skeleton.