Pigeon whole behavior is closely related to the loft environmental comfort and pigeon whole health. For human observation and recording the pigeon whole behavior is time-consuming, sampling limited, subjective and other issues, to timely meet the pigeon whole precision detection and pigeon whole behavior and health, based on the YOLO v4 pigeon whole behavior detection method was proposed. In this method, CSPDarkNet53 was used as the Backbone network to extract feature maps covering shallow semantic information of pigeons, and then PANet was used to transfer the bottom features and stack features to the top. Aiming at the high similarity degree of pigeon social behavior features, in order to achieve accurate identification of pigeon behavior in complex environment. The adaptively spatial feature fusion (ASFF) module was adopted to improve the YOLO v4 model, and the ASFF module was added to the feature pyramid network, which can adaptively fuse multi-layer features according to the feature weights and make full use of the features information of different scales. Moreover, ASFF can effectively filter spatial conflict information and suppress reverse gradient inconsistency, improve feature proportion invariance and reduce inference overhead. Based on the cleaning and social behaviors of meat pigeons in multiple periods, a database of five kinds of meat pigeon behavior images was made. OpenCV tool was used to process blur, brightness, haze and noise to expand the image data set (totally 10320 images), increase data diversity and simulate different recognition scenes, and improve the generalization ability of the model. A 8∶2 ratio was used to divide the training and validation sets. The training iterated 300 epochs in total, and the detection was carried out through meat pigeon data sets of different time periods, angles and sizes. The detection results showed that the detection accuracy of improved YOLO v4-ASFF model was 14.73 percentage points and 14.97 percentage points higher than that of mAP50 and mAP75 of original YOLO v4 model at the threshold of 0.50 and 0.75. Compared with Faster R-CNN,SSD, YOLO v3, YOLO v5 and CenterNet model, mAP50 of the YOLO v4-ASFF was improved by 13.98 percentage points, 14.00 percentage points, 18.63 percentage points, 14.16 percentage points and 10.87 percentage points in test set, respectively. The video detection speed was 8.1f/s, and the improved model had higher recognition accuracy under the condition of the same inference speed, strong generalization ability in complex environment, and less misdetection and omission of behaviors with high similarity. The research on meat pigeon behavior detection can provide technical reference for intelligent meat pigeon breeding and scientific management.