Above-ground biomass (AGB) is an important index to judge the growth and development of crops. Rapid, accurate and non-destructive remote sensing monitoring of AGB at different growth stages of crops is of great significance to precision agricultural production. A field experiment was carried out in Guanzhong area of Northwest China. Winter rapeseed under different water and nitrogen treatments was used as the research object. The multi-spectral images of winter rapeseed in vegetative and reproductive growth periods were obtained by UAV, and the AGB measured data of winter rapeseed were obtained by field experiment. The shadow and soil background in multi-spectral image were masked by threshold method, and the reflectance of each band was extracted to construct vegetation index. The correlation analysis between the measured data of winter rapeseed AGB and spectral variables was carried out, and the top eight spectral variables with the absolute value of correlation coefficient in each growth stage were selected as input variables. The AGB estimation model of winter rapeseed at different growth stages was constructed by random forest (RF), support vector machine (SVM), genetic algorithm optimized support vector machine (GA-SVM) and particle swarm optimization support vector machine (PSO-SVM) to determine the best estimation model. The results showed that the red band reflectance in the whole growth stage and reproductive growth stage was the most significant and stable, and the correlation coefficients were 0.835 and 0.754, respectively. The NBI in the vegetative growth stage was the most significant and stable, and the correlation coefficient was 0.846. The PSO-SVM was more suitable for the inversion of AGB at different growth stages of winter oilseed. The validation set R2 of the whole growth period, vegetative growth period and reproductive growth period were 0.866, 0.962 and 0.789, respectively.