To address the problem that the convolutional neural network models are difficult to be applied in practice due to their vast number of parameters and computation, a model compression method based on attention mechanism and dynamic sparse constraint was proposed. Firstly, the importance of each channel in the network was evaluated with the help of the squeeze and excitation networks (SENet) module, and sparse regularization was applied; then an adaptive penalty weight design method for network sparsity was proposed. According to the learning effect of the model, the weight was dynamically adjusted and added to the final training target to realize the dynamic compression of the model. Finally, the proposed model compression method was verified by experiments on the classic multi-classification dataset CIFAR-10. It was proved that the proposed model compression method based on attention mechanism and dynamic sparse constraint can reduce the network redundancy, resulting in a 43.97% reduction in the amount of network model parameters and an 82.94% reduction in the amount of computation, while the classification accuracy was only 0.04 percentage points lower than that of the original VGG16 model. Then the proposed model compression method was applied to the weed detection task, and the experiment was carried out on the sugar beet and weed datasets. The experimental results showed that compared with the unpruned model, the pruned model reduced the model parameters by 41.26%, the calculation amount by 45.77%, and the average detection accuracy by only 0.91 percentage points, which proved that this method could also have a good effect on the weed detection task.