In order to improve detection precision of crop phosphorus (P) nutrition level, in view of the problem that the present detection precision of crop phosphorus nutrition level based on spectral analysis is low and the spectral reflectance of phosphorus was influenced by both chlorophyll and anthocyanin, a phosphorus nutrition diagnosis strategy was proposed by fusing spectrum characteristics and physiological characteristics of tomato samples. With five levels (25%, 50%, 75%, 100% and 150%) of P nutrition stress samples cultivated by soilless cultivation mode as the research objects, reflectance spectra of different nutrient deficiency greenhouse tomato leaves was acquired by spectrum analyzer as well as the SPAD values of tomato leaves were obtained by SPAD-502. In addition, anthocyanin contents in leaves were determined. By using the spectral reflectance data under four characteristic wavelengths and physiological characteristics (anthocyanin content and SPAD value) as characteristic variables for tomato phosphorus nutrition diagnosis, the P nutrition diagnosis model was built based on least squares support vector machine (LS-SVM). An improved particle swarm optimization (IPSO)—adaptive inertial weight particle swarm optimization (AIWPSO) was designed to search the optimum values of SVM parameters for improving the search efficiency and avoiding getting lock in the local optimization. The proposed method with reflectance spectral and physiological characteristics (model 1) was compared with other three different models. For model 2, the method was same as the model 1 with the spectral features data only, model 3 was traditional LS-SVM which the optimum values of SVM parameters were obtained by cross validation of spectral and physiological characteristics data and model 4 was same as the model 3 with the spectral features data only. The results showed that the correlation coefficient and root mean square error of P were 0.9611 and 0.461, respectively, higher than those of other methods presented in the experiments. It can be concluded that the accuracy of P nutrition prediction model of tomato was improved by combing spectral characteristics with physiological features. The LS-SVM model with IPSO can acquire better parameters than traditional LS-SVM model based on cross validation. The combination of spectral and physiological characteristics data with the proposed algorithm was proved to be a powerful diagnosis tool for P nutrition status in tomato, and provided a new idea for the rapid detection of tomato P nutrient content.