In order to solve the problem that the traditional field obstacle recognition methods rely on manual feature extraction, long calculation time, and it's difficult to achieve real-time recognition in unstructured field environment, an optimized unstructured field obstacle instance segmentation method based on Mask R-CNN model was proposed. Firstly, an unstructured field obstacle dataset was constructed by aerial photography and network search. And then based on the ResNet-50 residual network, the spatial attention was introduced to focus on the significant apparent features of the tracking target, and the influence of useless features such as noise was suppressed. In addition, the deformable convolution was introduced into the structure of the ResNet-50 to add the offset, increase the receptive field and improve the robustness of the model. Comparative analysis was made by adding spatial attention and deformable convolution to different stages in the structure of ResNet-50. The results showed that compared with the original Mask R-CNN model, the mAP values of Bbox and Mask in Mask R-CNN improved by adding spatial attention and deformable convolution in Stage 2, Stage 3 and Stage 5 of the ResNet-50 were increased from 64.5% and 56.9% to 71.3% and 62.3%, respectively. The improved Mask R-CNN can well realize field obstacle detection and provide technical support for plant protection UAV to work safely and efficiently in unstructured field environment.