For automatic behavior monitoring of group-housed pigs in video surveillance, pig head/tail identification has important significance to improve the level of behavior recognition. A head-tail recognition algorithm was proposed based on YOLO v3 (You only look once v3) and pictorial structure models. Firstly, the object detectors of three categories, i.e., pigs, heads and tails, were trained with YOLO v3, which was a general object detection model based on deep convolutional neural networks. In this way, bounding boxes of pigs, heads and tails can be detected from the input image. Next, pictorial structure models were introduced to describe structural characteristics of heads and tails for pigs. For each detected bounding box of pigs, scores of all possible head-tail combinations were computed with the established pictorial structure model to choose the optimal part configuration. When a head or tail was missed in the pig bounding box, a part inference method based on threshold segmentation was utilized to estimate the missing part according to the major axis of the fitted ellipse. In experiments, an image dataset was constructed from a top-view surveillance video of group-housed pigs. Experimental results demonstrated that via the proposed method, the precision and recall of part localization were improved compared with results of YOLO v3. Moreover, the head/tail identification accuracy reached 9622%, which obviously outperformed other methods based on intersection of bounding boxes and generalized Hough clustering. As a result, the proposed method can effectively detect pigs and distinguish their heads/tails in images of group-housed pigs without excessive limitations on environments.