To reduce the environmental pollution and pesticide waste in orchards, a real-time method to detect the fruit tree leaf wall area (LWA) based on the improved YOLACT model was proposed to estimate average distance and density in the videos that captured by depth-color binocular camera, which can provide data for the real-time adjustment of pesticide spraying dose and spraying distance on intelligence pesticide spraying. Firstly, the YOLACT model was improved by using the ConvNeXt backbone network, and the NAM channel attention mechanism was introduced to optimize the model. Secondly, a leaf wall density estimation method based on deep learning was proposed. Finally, the average distance calculation method of LWA was proposed by excluding the interference information in the depth image through the threshold algorithm to simplify processing flow. The experimental results showed that the segmentation APall metrics of the improved YOLACT model was 91.6%, which was increased by 3.0 percentage points compared with that of the original model, and 2.9 percentage points, 1.2 percentage points and 4.1 percentage points compared with that of YOLACT++, Mask R-CNN, and QueryInst. The root mean square error (RMSE) of the leaf wall density estimation method was 1.49%, 0.82% and 2.20%. And the processing speed of the realtime LWA detection method could reach 29.96f/s.