Mastering the plot characteristics, spatial agglomeration, and drivers of nongrain production on permanent basic farmland can provide a reference for optimizing the spatial layout of crops and improving the fine control strategy of cultivated land. Based on the data of the Third National Land Survey of Guangxi Zhuang Autonomous Region, the current status, types, and plot characteristics of non-grain production on permanent basic farmland were analyzed. The spatial exploratory analysis method, ordinary least square, and geographically weighted regression models were used to identify the spatial aggregation characteristics, drivers, and the spatial variation in their effects of non-grain production on permanent basic farmland in the Guangxi Zhuang Autonomous Region. The results showed that the non-grain production plots of permanent basic farmland were mainly distributed in the central, western, and northern parts of the region, and the non-grain production types were mainly fruit trees and arbor, accounting for 88.85% of the non-grain production area. The quality and irrigation conditions of non-grain production plots were lower than that of the regional average of permanent basic farmland, but they were concentrated in areas with flat terrain, with 77.86% of plots with field surface slopes no more than 6°. The nongrain rate of permanent basic farmland in the counties of the region presented a significant positive spatial autocorrelation. The H-H agglomeration was mainly distributed in the central and southwestern parts of the region, the L-L agglomeration was mainly distributed in the eastern and northwestern parts of the region, and the H-L type was concentrated in the west. Field size, per capita permanent basic farmland area, per capita disposable income of rural residents, and distance from each county to the provincial capital were positively correlated with county-level permanent basic farmland non-grain production. Quality and irrigation guarantee rates were negatively correlated with non-grain production of permanent basic farmland in the county, with significant spatial variability in the impact of different drivers on non-grain production. In the future, the control of non-grain production plots of permanent basic farmland in the region should be strengthened in terms of optimizing crop layout, strengthening quality construction, regulating attribute characteristics, and improving protection compensation.