Taking agricultural vehicle with machine vision navigation as study object, a self-adaptive fuzzy control method with improved particle swarm optimization algorithm was designed. Firstly，by establishing 2-DOF steering model and visual preview model, lateral control equations of vehicle were described. Secondly，in order to improve the convergent speed of particle swarm optimization (PSO) algorithm，an improved PSO algorithm was designed. Finally, agricultural vehicle guidance system was a complex system with high nonlinearity, time-varying and large delay; therefore, an adaptive fuzzy controller was used for path tracking control. Correction factors were introduced into the fuzzy controller and particle swarm algorithm was used to optimize the correction factors. Taking the integral time absolute error (ITAE) sum of lateral offset and heading offset as the objective function, optimal correction factors were calculated by using PSO algorithm. Simulation and experimental results showed that the designed control algorithm could eliminate the lateral offset rapidly with less overshoot and rapid response. It retained the advantages of fuzzy control method and improved the control quality of guidance system. Compared with standard fuzzy control method, the improved fuzzy control method has a significant improvement on navigation accuracy under the same parameters condition. When the velocity of vehicle was 0.8m/s, the maximum lateral offset of straight path and curve path were less than 4.2cm and 5.9cm respectively, which could meet the requirement of agricultural vehicle navigation.