Currently the phenomenon of red-seed (the maturity of cottonseed is low) exists in the cottonseed processing. Most of the domestic cottonseed processing companies still rely on artificial monitoring to check the red-seed. However, visual inspection of workers exists some problems such as low automation level, large subjective factors and high labor costs. And breakage of cottonseed germination rate and other issues were resulted from the process of cottonseed processing. To solve these problems, a bilateral dual CCD (Charge-coupled device) detection and FPGA (Field-programmable gate array) sorting approach were presented. Through the analysis and design of the image acquisition module, data transmission module and sorting module, and combined with simulation and test trials, the effect of color sorter system was verified. Among them, the image acquisition module can acquire the maturity and integrity information of delinted cottonseeds by using color linear CCD and monochrome linear CCD sampling. Data transmission module is to complete the transfer task of data collection. Sorting module is based on cottonseed maturity and integrity information to sort discriminant delinted cottonseed quality. The results showed that color sorter system can complete the cottonseeds feature information collection, data transmission and sorting discrimination work, which met the color sorter design requirements.