The behaviors of cows, including eating, lying, standing, walking and tailflicking, which directly or indirectly reflects the health and physiological condition of the cows. It is necessary to monitor cow diseases and detect anomalies in cow behavior. In order to achieve the goals, a multi-branch parallel CAFNet (ConvNeXt-ACM-FAM) cow behavior recognition model was proposed by combining temporal and spatial attention information. The model combined an asymmetric multi-branch convolutional module (ACM) and a feature attention module (FAM) on the basis of a ConvNeXt convolutional network. Firstly, ACM was utilized to partition channel branches for feature extraction, and retained some original features to prevent excessive information loss. And ACM can improve the running efficiency of the model. Secondly, FAM fused the features from different channels and introduced the SimAM attention mechanism, which enhanced the efficient extraction of important features without increasing network parameters and improved recognition accuracy. The result of experiment demonstrated that the CAFNet achieved recognition accuracy of the method for eating, lying, standing, walking, and tailflicking was 95.50%, 93.72%, 90.26%, 86.43%, and 89.39%, respectively. And the average recognition accuracy was 91.06%. Compared with the original model, the number of parameters was reduced by 1.5×106, the computational complexity was reduced by 3×108, and the average recognition accuracy was increased by 8.63 percentage points. The results can provide technical support for cow disease monitoring and prevention.