It is of great significance to realize the intelligent cognitive decision-making ability of robots in the agricultural field and help the further development of smart agriculture that researchers use human cognitive experience and objective knowledge to assist computers and robots in object cognition and behavioral decision-making under the small sample data situation. On the prerequisites of the ability to recognize and judge basic attribute information such as image color and image shape by using methods such as statistical counting and support vector machine(SVM), tools such as Protégé was firstly used to build a professional knowledge base for fruit recognition and classification based on human cognitive experience and objective knowledge in object recognition. Then, under the rules set by artificial experience, the color information and shape information obtained from the image were used as the input of the knowledge base, and the classification results of the items in the image were obtained through matching reasoning. The experiments selected and used 2091 images from the Fruit360 public data set for the first part experiment，which included multiple fruit images of grapes, bananas, and cherries. The research firstly selected 30 images of grapes, bananas and cherries as the training set and validation set for the computers image attribute ability learning, and then the image classification performance was tested on the data set of the first part experiment. The experimental results showed that the image classification accuracy of grapes and cherries was 100%, and that of bananas was 93.30%. Subsequently, totally 984 yellow peach images in the Fruit360 public data set were selected as the data set for the second part experiment. By only adding the knowledge of yellow peach to the professional knowledge base built with ontology technology, the classification accuracy of the images can reach 97.05%. All experimental results showed that the proposed method can effectively accomplish the task of image classification decision-making and the method had good process interpretability, ability sharing and scalability.