Abstract:Aiming at the inherent limitations of manual growth stage judgment in industrial Tremella aurantialba (golden ear) cultivation, including strong subjectivity, low efficiency, and difficulty in standardizing evaluation criteria, a multimodal recognition method for golden ear growth stages was proposed by integrating visual and environmental information. Firstly, a multimodal dataset of golden ear was constructed, which incorporated synchronously collected images and key environmental parameters (temperature, humidity, light intensity, and CO2 concentration). This dataset covered 50420 samples across three distinct growth stages, namely the young ear stage, color conversion stage, and mature stage. Secondly, an improved Vision Transformer ( ViT) network was designed for image feature extraction, where multi-scale feature embedding, dynamic positional encoding, and a ResNet-based local perception enhancement module were introduced to enhance the modeling capability of texture details and spatial correlations of golden ear cultivation bags. Meanwhile, an adaptive weight fusion strategy for environmental factors was developed, which employed learnable weights to perform differential weighting on environmental parameters, thereby achieving joint optimization of visual and environmental multimodal features. Experimental results show that the overall recognition accuracy of the proposed method on the test set reaches 93.2%, with a macro-average F1-score of 93.3%. It outperforms the original ViT and ResNet50 baseline models by 7.2 and 8.7 percentage points in terms of F1-score, respectively. In particular, the F1-score for the young ear stage is as high as 98.8%. Ablation experiments further verified that the multi-scale feature embedding, local perception enhancement module, and environmental fusion strategy all contributed significantly to the improvement of model performance. Moreover, the importance analysis of environmental factors quantified the differences in environmental sensitivity among different growth stages of golden ear; notably, the color conversion stage exhibited the highest sensitivity to humidity (weight 0.40) and light intensity (weight 0.25). The research realized high-precision and interpretable recognition of golden ear growth stages. With only 52.1% of the parameters of the original ViT and an inference speed of 22.2f/s, the proposed model met the real-time monitoring requirements of industrial production, providing an efficient technical solution for the intelligent and precise management of industrial edible fungi cultivation.