It is easy to measure soil surface roughness by using a reference white board with a scale, from which the interface between soil and reference board can be detected and information of soil height can be read. Considering the low efficiency of manual reading, compute reading will be a good choice. But the impact of field illumination and weed interference makes compute reading susceptible. A soil roughness measuring method was proposed. In the proposed method, images were acquired with simplified reference scaling board. To process image automatically, color operation and threshold segmentation were used to decrease the effects of weeds and shadow, and then the soil boundary and scale were acquired, which would be used to measure the soil roughness. To improve the automaticity and robustness of the measuring method, chaotic particle swarm filter was applied for threshold segmentation. The test results showed that the soil roughness measurement method using color operation and chaotic particle swarm optimization reduced the requirement to image acquirement environment, and could calculate soil roughness quickly and efficiently, with the height error less than 0.5cm, the root mean square height error less than 5%, and the correlation length error less than 1%, which met the requirements of soil roughness real time on site measurement.