In order to reduce the influence of moisture and particle size on the soil organic matter prediction model established by the characteristic wavelengths selected in the traditional way, a method of extracting characteristic wavelengths was proposed. Sixty soil samples were collected from Shangzhuang Experimental Station of China Agricultural University, and the samples were naturally dried and divided into two, one portion was formulated into five particle size gradients (particle size of 2~2.5mm, 1.43~2mm, 1~1.43mm, 0.6~1mm, and 0~0.6mm), the other part was sieved through 0.6mm and formulated into five moisture gradients (5%, 10%, 15%, 20%, and 25% moisture content). The true values of soil organic matter content and soil spectral information were obtained by standard instruments, and the characteristic wavelengths were extracted by using the random frog-hopping algorithm. Totally seven wavelengths with high correlation with the true values of soil organic matter content were selected as the characteristic wavelengths under each moisture and particle size gradient, and multiple linear regression (MLR), partial least squares (PLS) and random forest (RF) models were established respectively. The results showed that the R2 of the modeling and prediction sets of the three models basically tended to decrease as the water content increased;the R2 of the modeling and prediction sets of the three models was the lowest in the gradient of 2~2.5mm, highest in the gradient of 0~0.6mm, and close to the R2 of the modeling and prediction sets in the rest of the gradient. Combined with the filter bandpass range of ±15nm, eight characteristic wavelengths of soil organic matter under moisture gradient were selected as the same or close to each other, and six characteristic wavelengths under particle size gradient were selected, and finally six wavelengths were eliminated under the 14 characteristic wavelengths determined under moisture gradient and particle size gradient by combining chemical bonding characteristics, and eight characteristic wavelengths were determined as follows: 932nm, 999nm, 1083nm, 1191nm, 1316nm, 1356nm, 1583nm, and 1626nm. The MLR, PLS and RF models were established respectively, and the results showed that the R2 of the modeling set and the R2 of the prediction set were not less than 0.8 and 0.75 for the three models established by the final selected organic matter characteristic wavelengths, and the best prediction effect was achieved by PLS, with the R2 of the modeling set and the R2 of the prediction set being 0.8809 and 0.8402, respectively. The model established had better applicability and prediction effect, and the influence of moisture and particle size on prediction was eliminated to a certain extent compared with the traditional way.