To comprehensively assess the potential of integrating unmanned aerial vehicle (UAV) remote sensing and multi-temporal parameters fusion in predicting winter wheat yield in the past, the RGB and multi-spectral data from UAVs spanning seven critical growth stages of winter wheat were collected. From these data, spectral and morphological parameters were directly extracted. Five machine learning algorithms were then employed to compare and evaluate the yield prediction performance at each individual growth stage. Subsequently, an in-depth analysis was conducted, based on the identified optimal parameter combinations, to examine the relationships between various growth stages and the accuracy of yield predictions. The results revealed that both individual growth stages and their combinations significantly impacted the prediction of winter wheat yield. Among the single growth stages, the filling and flowering stages achieved the highest prediction accuracy, followed by the heading, booting, maturity, jointing, and tillering stages. When considering multiple growth stages, the prediction accuracy was progressively increased from dual-stage to tri-stage and quad-stage combinations. However, balancing the marginal gains in accuracy against factors such as data acquisition and processing costs, as well as computational resources, the tri-stage combination of “jointing + heading + filling” emerged as the most cost-effective solution. In terms of the five machine learning algorithms employed, the overall prediction accuracy ranked from the highest to the lowest was as follows: BPNN, RF, SVM, XGBoost, and SMR. Notably, while the optimal combinations of spectral and morphological parameters identified through the SHAP method varied across growth stages, they consistently enhanced the yield prediction accuracy for all stages excepted the jointing stage. The research result can provide valuable methodological insights and technical references for the precise prediction of winter wheat yield per unit area in the past.