In recent years, deep learning has gradually developed in flower recognition research, which has a positive impact on the growth management and fruit production of orchard fruit trees. In order to tackle the problem that the densely gathered litchi flowers cannot be recognized by instance segmentation method in natural environment, a deep semantic segmentation network was proposed to recognize and segment flowers and leaves pixels. Firstly, pictures of litchi flowers were shoot in the experimental orchard in the flowering stage, which were taken to make pixel-level images, and then were used for data augmentation. Then a backbone network of 34 layers based on ResNet was built, in which dense features were connected layers by layers and in order to exploit the useful information, attention blocks were also added into the networks. A dense features connection method meant each layer was connected to every other layer in a feed-forward fashion, different from that features were only from the last consecutive layer. Attention block was a mechanism of propagating information useful for the specific task and suppress the useless one. Finally, a full convolution layer was added for image pixel prediction. The average intersection union ratio of the proposed model was 0.734, and the pixel recognition accuracy reached 87%. In summary, with good robustness and high recognition accuracy, the proposed deep semantic segmentation model can solve the problem of litchi flower recognition and segmentation, and provide visual support for intelligent flower thinning.