Rice grain detection plays an important role in grain storage, directly affecting the price of grain sales. In response to the problems of difficult recognition, large network model parameters, slow detection speed, and high cost of general machine vision detection algorithms in dense scenes with small rice grain targets, a rice grain detection algorithm was proposed based on YOLO v7 optimization. Firstly, some efficient layer aggregation network (ELAN) modules were replaced with lightweight network module GhostNetV2 and added them to the backbone and neck network sections, achieving precise simplification of network parameters while reducing feature redundancy in channels. Secondly, the attention module (ACmix) that combined convolution and self attention was added to the MP module, balancing global and local feature information and fully paying attention to the details of feature mapping. Finally, wise intersection over union (WIoU) was used as the loss function to reduce penalty term interference such as distance and aspect ratio. The design of monotonic focusing mechanism improved the positioning performance of the model. The improved model detection level was verified on the rice grain image dataset, and the experimental results showed that the improved YOLO v7 model was high, mAP@0.5 was up to 96.55%, mAP@0.5:0.95 reached 70.10%, and the training model parameters were also decreased. In practical scenarios, the effect of rice impurity detection with a dark black background was better than other models, meeting the real-time detection requirements of rice grains. This algorithm can be considered for application in automated grain detection systems.