Aiming at the significant symptom of broilers respiratory disease like Dyspnea, an early detection of broilers respiratory disease based on YOLO v5 and short time tracking was proposed. After the specific optimization of YOLO v5 algorithm, such as the adaptive setting of anchors and the application of CIoU Loss (Complete IoU Loss), the broiler heads can be accurately identified in the complex environment and whether it was in the open-mouth state can be detected at the same time. According to the intersection over union with heads coordinated from different frames, different broiler heads can be tracked in short time, and the action sequences of different chicken heads can be obtained. Then the action sequences can be analyzed to judge the frequency of mouth-opening and mouth-closing combination to detect the Dyspnea dynamically. The experimental results showed that the mAP of the improved YOLO v5 for broiler heads was 80.1%, the accuracy of mouth-opening head was 67.3%, and the accuracy of mouth-closing head was 92.8%. The recognition accuracy of the Dyspnea detection method based on time series was 91.8%, the recall was 75%, and the precision was 67.9%. The method proposed can help to early detect the broilers respiratory diseases in the group breeding environment.