Aiming at the problems of the existing remote sensing yield estimation methods that do not model the dependence between channels and ignore integrating other features outside the image, an interannual crop yield estimation method based on CNN-S-GPR was proposed for hyperspectral images, taking Ningxia wolfberry yield as an example. Firstly, histogram statistics, histogram normalization and time series fusion were used to construct the data set, which realized the fusion of multi-band and multi-temporal images. Secondly, using convolutional neural networks to extract features from the data set; and then the channel attention mechanism was used to characterize the importance of different channels. Finally, Gaussian process regression (GPR) was introduced to explicitly integrate image features and spatial location features further improved the accuracy of production estimation. The test results showed that compared with that of other yield estimation models, MRE and RMSE of this model were decreased from 0.44 percentage points to 0.95 percentage points and from 52.48t to 82.65t, respectively, and the coefficient of determination reached 0.91. It realized the complex fitting of the output of wolfberry in 16 counties of Ningxia during the year, which was of great significance to the agricultural planning layout, policy adjustment and sustainable development.