Aiming to address the issue of low targeting accuracy in controlling the unloading arm position during rice transfer from unmanned rice harvesters to grain transport vehicles, which relies on Beidou positioning information of the harvester and transport vehicle, a GTSM network for visual segmentation of grain compartment images was proposed to provide positional reference information for the unloading arm. Based on the DeepLabv3+ architecture, the lightweight ShuffleNetv2 backbone replaced Xception, and the atrous convolutions in the ASPP module were replaced with depthwise separable convolutions, followed by low-rank decomposition into micro-factorized convolutions to reduce model complexity and improve inference speed. Additionally, an SE channel attention mechanism was introduced in the shallow feature branch to enhance the model’s ability to utilize low-level features such as grain compartment edges and textures. Experimental results showed that GTSM achieved a mean intersection over union (mIoU) of 96.06% and a mean pixel accuracy (mPA) of 98.69%, representing improvements of 0.78 and 0.67 percentage points, respectively, over the baseline DeepLabv3+. Meanwhile, model complexity was significantly reduced, with parameter count and memory usage reduced to 1/9 of the original, and inference speed was increased by 166%. These results demonstrated that the proposed GTSM balanced segmentation accuracy and inference speed, providing a reference for automated grain compartment segmentation in field grain transport vehicles.