Each apple is unique but can be classified into an “apple type” via features such as color, contour, texture, and other physical characteristics. Many apple growers classify apple types manually, often at great expense due to misclassification errors, low efficiency, inconsistent results, and high labor costs. Therefore, a real-time apple type detection and classification system is needed to prevent these complications, which typically happen in the period between sourcing and sales. To automate apple type classification, EBm-Net, an automatic identification and classification model was proposed based on a dual-branch structure network. The model fully extracted the contour, color, and texture characteristics of an apple’s surface by fusing channel attention and spatial attention mechanisms; this was done to further increase the feature difference between apple types by using a distance metric. The effectiveness of the EBm-Net apple type classification method was validated by analyzing its feature map and category probability statistics map. Experimental results showed that the classification accuracy of the EBm-Net model applied to Red Fuji, Jonagold, Qin Guan, Xiao Guoguang, Golden Crown, Granny Smith, and Gala apples was 96.25%, 96.25%, 100%, 92.50%, 98.75%, 100% and 93.75%, respectively; the overall classification accuracy of the seven apple types was as high as 96.78%. Therefore, it was feasible to use visual images combined with deep learning to classify and recognize apple type, which provided a method for real-time autonomous apple type classification.