In view of the problems of high computational complexity, large memory consumption, and difficulty in deployment on mobile platforms with limited computing power in DeepLabV3+ segmentation model, an improved lightweight DeepLabV3+ algorithm was proposed to realize the segmentation and recognition of buckwheat by UAV at seedling stage. The algorithm adopted the fusion of re-parameterization visual geometry group (RepVGG) and mobile vision transformer (MobileViT) modules to establish the backbone network for feature extraction. At the same time, the squeeze-and-excitation networks (SENet) attention mechanism was introduced into the RepVGG network structure to capture more global semantic information by using the correlation between channels, and ensure the performance of buckwheat segmentation. Experimental results showed that compared with fully convolutional networks (FCN), pyramid scene parsing network (PSPNet), dense atrous spatial pyramid pooling (DenseASPP), DeepLabV3, and DeepLabV3+ models, the improved algorithm proposed greatly reduced the model parameters, making it more suitable for deployment on mobile terminals. The mean pixel accuracy (mPA) and mean intersection over union (mIoU) on the selfbuilt buckwheat segmentation dataset were 97.02% and 91.45%, the overall parameters, floatingpoint operations (FLOPs) and inference speed were 9.01×106, 8.215×1010 and 37.83 f/s, respectively, with the best performance. In the full-size image segmentation, the mPA and mIoU for buckwheat segmentation can meet the requirements at different flight heights, which had good segmentation ability and inference speed. The algorithm can provide technical support for the later buckwheat seed replacement, fertilization maintenance, and growth monitoring, and promote the intelligent development of small and coarse grain industry.