Five different varieties of tea samples were classified with the method combining stacked auto-encoder (SAE) with low-rank matrix recovery (LRMR). The hyperspectral imaging system with spectrum range of 431~962 nm was used to collect five kinds of tea samples containing 618 bands of hyperspectral images, including Huangshan green tea, Longjing, Yixing Mao Feng, Yunwu green tea and Biluochun. The ENVI software was used to determine the region of interest (ROI) of the hyperspectral images, and the average spectral data of the sample in ROI were extracted as the raw spectral information of the sample. Due to the large amount, strong redundancy and much noise of the hyperspectral information, the low-rank stacked autoencoder (LR-SAE), which combined the stacked auto-encoder and the low-rank matrix recovery, was used to reduce the dimensionality of the hyperspectral data of the tea samples. Support vector machine (SVM) and Softmax classification model were applied to identify the tea samples after dimensionality reduction. The 5-fold cross validation experiment results showed that the accuracy of prediction set of LR-SAE-SVM model was 99.37%, and that of SAE-SVM model was 98.82%. As for Softmax classification, the accuracy of prediction set of LR-SAE-Softmax model was 99.04%, and that of SAE-Softmax model was 97.99%. The results showed that the accuracy of the classification model based on LRSAE was improved on some degree and the LRSAE had better robustness than the traditional SAE without denoising. It was feasible and efficient to apply the classification model based on LR-SAE into tea variety identification.