To address the issue of estimating tractor side-slip in hilly and mountainous terrains, a multi-sensor information fusion algorithm that integrated machine vision and the global navigation satellite system (GNSS) was proposed. Initially, a simplified kinematic model of the tractor was presented, followed by separate discussions on skid estimation methods based on GNSS and machine vision technologies. The feasibility of the skid estimation methods was validated through joint simulation using CarSim and Simulink. Kalman filtering and weighting functions were introduced to dynamically fuse and adjust sensor data. An experimental platform mimicking hilly terrains was set up to conduct tests under varying road slopes, GNSS coverage conditions, and road surface conditions. The experimental results showed that under dry road conditions and GNSS blockage, the total skid amounts for tractors driving on 9° and 18° slopes were 0.322m and 0.432m, respectively, with relative error of 7.86% and 6.00%, which indicated that accurate skid estimation was still achievable even when GNSS signals were obstructed. The research result can provide methods and experimental foundations for precise lateral control of tractors.