In order to get spatial information of planting in time, protect and utilize Erigeron breviscapus, the fuzzy inter-ridge boundary and the difficulty in obtaining training data set of fine markers were solved. An unmanned aerial vehicle remote sensing planting information extraction method for Erigeron breviscapus based on the combination of RGB band maximum difference method and weakly supervised semantic segmentation was proposed. Firstly, Erigeron breviscapus was labeled at border level in order to make weakly labeled data set to reduce labeling time cost. Then, a lightweight U-Net network was used for weakly supervised semantic segmentation of the weakly labeled data set to achieve rough extraction of Erigeron breviscapus. Finally, the RGB band maximum difference method was used to remove the nonErigeron breviscapus in the rough extraction results to achieve the fine extraction of Erigeron breviscapus growing area. The experimental results showed that the proposed method in IoU was 90.55%, 90.74% and 86.63%, respectively, in three selected Erigeron breviscapus scenes, and the accuracy was higher than object-oriented classification method and maximum likelihood method. The effectiveness of the method was verified by ablation experiments.