Correctly identifying the field location of broccoli is the basis for realizing automatic harvesting of broccoli. Because the flower ball color is similar to the plant stem, broccoli cannot be identified only by color features. The algorithm firstly strengthened the boundary texture of the image through pretreatment and Laws filter, in which the filter kernel function of Laws adopted E5×E5. Then Gabor filter was applied to the texture enhanced image, and Gabor transform which was a short-time window Fourier transform proposed to meet the locality of two dimensional images in spatial and frequency domain, with window function of Gaussian function. Through Gabor filter, each pixel had a 1×8 dimensions texture feature vector, which was generated by eight different Gabor filtering kernel functions that were determined by the wavelengths of one sinusoidal modulation wave and the directions of eight different kernel functions. The texture feature vector was zero-mean normalization to speed up the convergence of clustering process, and K-means clustering segmentation and open operation were performed to obtain the potential region of broccoli heads. Meanwhile, the image was segmented based on color features. Through converting RGB (red, green, blue) image into HSV (Hue, Saturation, Value) image, the Hue component of the image was threshold to filter out ground pixels. Finally, the results of texture feature recognition and color feature recognition were fused to realize the recognition of field broccoli heads. A total of 792 images were used for the experiment. The experimental results showed that this method could accurately identify the broccoli field images. The precision rate was 96.96%, the recall rate was 94.41%, and the F1 score was 95.67%. Through the algorithm recognition of three sets of different shooting environment data sets, the F1 score of the three sets of data sets was always maintained at more than 94%, which had good shooting environment adaptability and laid a foundation for automatic harvesting of broccoli by agricultural robots.