The quality difference of peanuts from different origins is significant, and it is common to see inferior peanuts being sold with high-quality labels. Therefore, it is crucial to provide a peanut origin traceability method. A bimodal fusion feature attention (DFFA) was proposed based on electronic nose and hyperspectral system for non-destructive detection, and DFFA-Net was designed to achieve peanut quality identification. Firstly, the gas information and spectral information of peanuts from seven different origins were obtained by using an electronic nose and hyperspectral system. The gas information from the inside out of peanuts can characterize their overall macroscopic quality, while the spectral information differences of different chemical bonds and functional groups can characterize their overall microscopic quality. Then, DFFA was proposed to adaptively fuse the gas-spectral dual-modal information and focus on important features that affected classification performance. The necessity of fusing dual-modal information was verified through ablation experiments. Finally, based on the proposed DFFA module, DFFA-Net was designed with optimized network structure to achieve effective identification of peanut quality from different origins. Through ablation analysis and comparison of classification performance with multiple attention mechanisms, DFFA-Net achieved the best classification performance: accuracy of 98.10%, precision of 98.15%, and recall of 97.88%. The effectiveness of DFFA-Net in peanut origin identification research was validated. In conclusion, the proposed DFFA-Net, combining electronic nose and hyperspectral system, effectively realized the quality identification of peanuts from different origins and provided an effective technical method for quality supervision in the peanut market.