In sericulture, cocoons must be detected and classified before silk reeling is performed. It is important for improving the quality of silk to eliminate the yellow spotted cocoons. A large number of checker cocooning are used for silkworm mounting cocooning frames. It is difficult to detect the yellow spotted cocoons because of the checker cocooning frames’ reshape and outer floss. To solve the problem, the algorithm based on FCM and HSV color model was used to detect and eliminate the yellow spotted cocoons in the cocoons harvested process. Firstly, FCM segmentation was applied to the original image of the checker cocooning frame to eliminate the outer floss and the frame. The binary image of the cocoon was obtained by FCM segmentation and threshold segmentation. The original image was masked with the binary image which was obtained by FCM segmentation. And the individual cocoon was extracted through the masked operation. According to the proportion of specific color components in the color histogram which was gotten by accumulating color of HSV, the yellow spotted cocoon was judged one by one. The center point coordinates of the yellow spotted cocoons’ regions were got by the connected components calibration, and were mapped into the world coordinates through the equation that image coordinates to world coordinates to get the cocoons positions in the Cartesian space. Finally, the yellow spotted cocoons were eliminated by automatic harvesting machine. According to the result of experiment, the correct ratio of cocoon detection was 81.2%, the location accuracy was 3.0mm, the average process time of one mountage image was 1.271s. The cocoons which out floss with leaf stalks or crushed leaves could be detected errorlessly, but the algorithm had no effect on detection the cocoons with stained point in the edge.