How to assess the number of rice panicles had been one of the key ways to realize high-throughput rice breeding in the modern smart farming, for that the panicle can reflect rice yield directly. In practical in-field scenarios of rice growing, the size of panicles was relatively small while the panicles were always occluded by the leaf seriously. So, it was a challenging task to accurately identify the rice panicle in the complex field scene and automatically count the number of panicles. In order to count the small-sized rice panicles locally occluded by leaves, an automatic counting method was designed which called generative feature pyramid for panicle detection (GFP-PD) based on the feature pyramid and the generative adversarial networks. To solve the problem of feature loss in feature learning of small size rice panicles, firstly, the relationship between the size of rice panicle and receptive field was analyzed quantitatively, and then the appropriate feature learning network was selected to reduce the information loss of rice panicles;secondly, the multi-scale feature pyramid was constructed and integrated to enhance the panicle features. For the noise in the panicle feature which caused by the leaves occlusion, a feature repairing network which called occlusion sample inpainting module (OSIM) was designed to optimize the quality of features containing leaves noise by restoring the noise to the real feature of rice panicles. The model was trained and tested by the in-field rice images taken from the variety of Nanjing 46. The average panicle counting accuracy and the average panicle recognition accuracy of GFP-PD were 90.82% and 99.05%, respectively, which were 16.69 percentage points and 5.15 percentage points higher than the results of Faster R-CNN. When constructing the feature pyramid for Faster R-CNN, the average counting accuracy and recognition accuracy based on VGG16 network were 87.10% and 93.87%, respectively, which were 3.75 percentage points and 1.20 percentage points higher than ZF network. After the OSIM repairing model was further used to optimize the panicle feature, the recognition accuracy was increased from 93.87% to 99.05%. The results showed that selecting the appropriate feature learning network and constructing the feature pyramid could significantly improve the count and recognition accuracy of small-size rice panicles in the field. The OSIM can remove the leaf noise in the feature of rice panicle effectively, which was useful to improving the recognition accuracy of the panicles partially covered by the rice leaves.