An algorithm was proposed to on-line identify the defects and stem-calyx on apples based on lightness correction method and AdaBoost classifier. The ‘Fuji’ apples were selected as the experiment object. First, the RGB images and NIR images of apples were acquired, and NIR images were binarized to obtain the mask images. Second, the R component images were corrected by using proposed lightness correction algorithm and the defect candidate regions were obtained by binarizing the corrected images with a single threshold. Third, every candidate region was treated as a mask, and the information of random seven pixels in the candidate region were selected as the characteristics of the selected candidate region. Finally, an AdaBoost classifier was used to classify these candidate regions by voting way, and the category of candidate region was determined according to the final voting results. For the investigated 140 samples, the results with a 95.7% overall detection rate under acquisition speed of three apples per second indicated that the proposed algorithm was effective.