To provide farmers with vegetable diseases knowledge, this paper proposes a method based on voice recognition technology to label and split vegetable diseases videos. Through this method, videos about vegetable diseases can be split into several smaller segments which are more suitable for cell phone. The methods of semantic annotation and video shot clustering were based on video segmentation and voice recognition. In this method, the audio signals of videos were transformed into text strings firstly by voice recognition. Then key semantic entities for labelling video shots semantically were split from the text strings. Finally different video shots were clustered and recombined based on keywords provided by user and the semantic labels of video shots. When applying the method of semantic annotation to two videos, the recall ratios were up to 96.08% and 94.93%, the precision ratios were up to 94.31% and 95.98%, and the F-1 measures were up to 0.93 and 0.92. As for method of video shot clustering, the recall ratios were up to 94.9% and 98.7%, and the precision ratios were up to 92.1% and 90.2%. Results of comparative experiments show that the proposed method is valuable both in theory and practice.