The control system is one of essential constitute parts of the fruit harvesting robots, which affects the work efficiency and quality of the robots. In generally, fruit harvesting robots are nonlinear spatial open chains with multiinput and multioutput and serious coupling. It is difficult to set up accuracy mathematical model for the joint control system because there are many uncertain factors during working, such as joint friction, assembly errors, etc. Sliding mode control method has more advantages to solve nonlinear problems with strong robustness, antidisturbance, etc. To improve the control performance of the fruit harvesting robot and solve the vibration of the control system, the sliding mode control (SMC) method based on genetic algorithm (GA) optimization was proposed. In this method, the parameters of SMC algorithm were optimized by GA through adjusting the parameters of switching function and exponential approach law in real time. Simulation and experimental platforms of a joint control system were designed and manufactured based on STM32 microcontroller, AS5045 position feedback module and CAN bus communication module. Moreover, experiments on position tracking and joint control system response were conducted in unloading and loading conditions respectively. The results show that the response velocity of position tracking was increased greatly by adopting GA. The vibration amplitude and time of the control system caused by external disturbance and load changes were reduced significantly. The experiment results also showed that the response time during position tracking of joint No.6 in the unloading condition is 0.3s shorter than that in the loading condition. However, the position tracking accuracy and maximum deviation are not affected greatly by adding load. In addition, the controller response time in actual experiments increases 0.5s comparing with that of simulation experiment owing to ignoring some uncertain factors, such as joint friction, assembly errors, inertia force, etc. in the simulation experiment system. But the control system can make the joint to track the expected trajectory with satisfied accuracy and high robustness. The results can provide the foundation for the further research of the robot joints’ control systems.