A fast segmentation method for high-resolution color images of cotton foreign fibers was proposed. It detected the edge of color image by using edge detection method based on improved mathematical morphology. A color image was converted into a gradient map in the first step, and the best threshold of the gradient map was chosen by selecting the best experience value iteratively. The experimental results indicated that the proposed method could segment the high-resolution color images of cotton foreign fibers directly and precisely, and the speed of image processing increased more than two times than the traditional methods.