The robotic egomotion and the motion of moving obstacle were overlapped when an agricultural mobile robot need to detect the moving obstacle based on machine vision. So two images were taken from the mobile robot and the Harris feature points were extracted and matched. Then a bilinear model was applied to model the movement between the two images, and a least square optimization method was used to calculate the model parameters. A transformation matrix was obtained with this model to compensate the first image to eliminate the effect of the egomotion of the mobile robot. Finally, a frame difference between the compensated image and the second image was carried out to detect the moving obstacle in the environment. Experimental results showed that this algorithm could eliminate the image movement caused by the egomotion of the mobile robot, and the moving obstacles were able to be detected effectively with machine vision for the agricultural mobile robot.