To effectively utilize the spectral and spatial information of limited labeled training samples in hyperspectral image (HSI) classification, a HSI classification approach with small sample set based on adaptive dictionary was proposed. Firstly, discriminating pixels of each labeled sample were extracted from spatial information with entropy rate segmented superpixels and spectral neighborhood, the training set was then extended by adding the discriminating pixels. Furthermore, the spatial-spectral information of each test sample was analyzed, and its adaptive dictionary was constructed by simplifying the extended training sample set. Finally, the spatial-spectral reconstruction was performed on the adaptive dictionary of each test pixel, where the collaboration and competition among dictionary elements were both considered. To evaluate the performance of the proposed approach, it was compared with some traditional methods by using spectral information and the state-of-the-art methods incorporated traditional information of fixed window size, experimental results on Indian Pines dataset with only 2% training set demonstrated that the overall accuracy of the proposed approach was 91.45%, which was 3.48~39.52 percentage points higher than that of other methods, and the results on Pavia University HSI with 1% training set showed that the overall accuracy of the proposed approach reached 95.54%, which was 2.45~21.63 percentage points higher than that of others, indicating the effectiveness of the proposed approach.