The optimization method of vortex pump design was established based on experimental design theory and the response surface approximation. Taking the maximum efficient, high effective area and stability of head-flow curves in the small flow region as the objective functions, the structural parameters of model pump were screened by Plackett—Burman experimental design method. The selected parameters were divided into three levels: significant factor, the second significant factor and non-significant factors, according to their influences on objective functions. Finally, the optimal design points for each level of structural parameters were determined by central composite design, Box-Behnken design and the response surface analysis method. Based on the computational fluid dynamics (CFD) results, the response surface approximation model for the pump structural parameters and multi-objective function were built to analyze the interaction between the effects of structural parameters. Furthermore, experimental research was done for the pump at the optimum working point. It was found that experimental results correspond well with the response surface approximation, with a relative efficiency deviation of 4.89%. The optimized pump improved significantly in performance than before. Thus, it is concluded that the response surface optimization design method based on experimental design theory can be used to optimize the design of vortex pump.