In order to solve the problem of forage recognition under natural conditions, a convolutional neural network method based on double-pooling feature weighting and multi-scale convolution kernel feature weighting structure was proposed. The spatial information and significance information of the image were fully utilized by using the dual-pooling feature weighted structure. Two groups of feature graphs were obtained by max-pooling and mean-pooling of feature graphs output from the convolution layer, and then these two groups of features were spliced. Finally, a feature re-calibration strategy was introduced to weight the importance of current tasks according to the feature graphs of each channel, so as to enhance useful features and suppress useless features. Image information was more fully mined by using multi-scale feature weighting structure. The 3×3 and 5×5 convolution kernels were used at the same time, and the features of the first several layers of the network were spliced with the features of the current layer to improve feature utilization rate. Feature re-calibration strategy was also introduced to weight features. The recognition experiments of ten pasture images showed that the recognition rate of the method was 94.1%, which was 5.7 percentage points higher than that of VGG-13 network, the double pooling and multi-scale feature weighting structure effectively improved the recognition accuracy.