A strong correlation exists between fertilizer application and grain yield. Due to many factors affecting grain yield, the existing fitting methods of correlation between the two variables lead to large errors. Aiming at the data of fertilizer application and grain yield in Taihu Lake Basin, the back propagation (BP) neural network was used in this paper to model the correlation between the two variables accurately, which could guide to reduce use of fertilizer. This paper collected average fertilizer use and grain yield data per acre in 35 years i.e. from 1980 to 2014, in 16 counties and cities in Taihu Lake Basin. Missing items were filled automatically through a time series analysis approach called auto-regressive and moving average model (ARMA). For average grain yield data, ARMA(2, 6) model had higher accuracy with mean square error (MSE) less than 0.2 and R2 more than 0.85. For average fertilizer use, ARMA(3, 7) model had higher accuracy with MSE less than 0.02 and R2 more than 0.80. Then BP neural network with a single hidden layer (1-10-1) was established to fit correlation fertilizer use and grain yield data in each country. Goodness of the fit with BP neural network was better than other methods, with MSE less than 0.12 and R2 more than 0.80. Results indicate that there is a threshold for fertilizer use. When fertilizer is used less than the threshold, grain yield per acre is more, whereas when it is more than the threshold, grain yield per acre fluctuates and the average keeps invariant. The correlation implies excessive application of fertilizers can not achieve high yields.