In order to accurately obtain the field crop planting distribution information to satisfy the needs of refining the management of agriculture, a field crop classification method was proposed for unmanned aerial vehicle (UAV) multispectral remote sensing images based on DeepLab V3+ network. In which the structure of the input layer was modified to fuse multispectral information with the prior features of vegetation indexes, and the activation function of Swish was adopted to maintain the backpropagation capability of the model when the response was a negative value. The research region was Shahaoqu irrigation field in the Hetao Irrigation District, Inner Mongolia Autonomous Region, whose UAV multispectral remote sensing images collected in 2018 and 2019 were taken as samples. The classification model was constructed and trained on the data of 2018, and the generalization performance of the model was tested on the data of 2019. The experimental results showed that the improved DeepLab V3+ model got excellent classification with fast speed. Its mean pixel accuracy and mean intersection over union were 93.06% and 87.12%, respectively, which were 17.75 percentage points and 20.8 percentage points higher than those of the traditional support vector machine (SVM) method using artificial features, and 2.56 percentage points and 2.85 percentage points higher than those of the original DeepLab V3+ model. Therefore, this method can learn more expressive semantic features from the field crop remote sensing images, thus obtaining accurate crop classification. The research result provided a new technical basis for the interpretation of farmland types using UAV remote sensing images.