Aiming to solve the problem of image jitter of the tracked agricultural robot on the bumpy road in the field. Firstly, the field jitter video image sequence was obtained by the camera, the image was divided into molecular regions, and the mean square error of gray value of each region was calculated, and then the Harris corner threshold of each region was determined, and the adaptive corner threshold was set. The corner distance constraint was added to complete the corner detection of the image. Secondly, optical flow tracking was performed on the detected corners, and the parameters of interframe motion estimation were calculated. Finally, the motion estimation parameters were smoothed by the adaptive Kalman filter algorithm, and the smoothing performance of the filter was dynamically adjusted to obtain the accurate motion estimation vector. The experimental results showed that the improved Harris corner detection algorithm reduced the standard deviation of the average distribution of the region. Under the premise of ensuring smooth random motion, the tracking performance of active motion of adaptive Kalman filter was improved by 30.75 percentage points. After image stabilization, the signal to noise ratio of the image was improved by 15.93%, and the processing time of single frame was 25.66ms, which can meet the realtime processing at the acquisition rate of 30f/s. The traditional Harris corner detection algorithm was improved to overcome the phenomenon of uneven corner distribution and easy clustering. An adaptive Kalman filter algorithm was proposed to suppress the random motion of the camera and improve the performance of tracking the active motion of the camera, which had a good image stabilization performance in tracked agricultural robots.