In crop classification with hyperspectral images, in order to make full use of the complete spectral information of hyperspectral remote sensing images and avoid the Hughes phenomenon caused by high-dimensional data, traditional methods usually adopt the strategy of “feature reduction first, and then classification”. Starting from the data dimensionality reduction of the autoencoder and the classification advantages of CNN network, the commonalities of the two networks in the training process was firstly analyzed, and a fusion network for hyperspectral image classification was constructed based on the selection of classifiers in the optimization process of the autoencoder. Compared with the traditional methods, this method can realize the direct classification of hyperspectral images through once supervision training, which simplified the traditional data processing process and had better classification performances. In the experiment, two sets of typical hyperspectral remote sensing image data sets from Pavia University and Xiong'an area were used to verify the method. The experimental results showed that in Pavia University dataset, when only 10% of pixels were selected as the training set, the overall classification accuracy of the proposed method reached 98.73%, which was more than 8 percentage points higher than those of the traditional method. In Xiong'an dataset, when only 1% of pixels were selected as the training set, the overall classification accuracy of this method reached 98.04%, which was more than 7 percentage points higher than those of the traditional method, which proved the correctness of this analysis and the effectiveness of the proposed method, and also provided a strategy for hyperspectral image classification with small training samples.