Recognizing and positioning the picking points (spatial position and coordinate points), and determining the appropriate picking pose according to the direction of fruit stem, are the keys for the robot to achieve efficient and lossless picking during harvesting. However, the harvesting environment is complex and changeable, the color of fruit stem is similar to the branches and leaves, and the tomato clusters are always with different colors and shapes. Furthermore, tomato clusters grow in different directions, and the end effector frequently interferes with the leaves and vine during picking, there are often situations of “not picking when robot see it”, which reduces the recognition accuracy of picking points and picking rate. Aiming at this problem, considering the growth characteristics of tomato clusters, a method for visual positioning and picking pose estimation of tomato clusters based on instance segmentation was proposed. Firstly, based on the instance feature standardization, and the mask scoring mechanism of the YOLACT algorithm, the high quality and reliable region of interest (ROIs) and masks of tomato clusters were collected. Specifically, in order to efficiently achieve the coarse segmentation of fruit stems via the YOLACT. Then, according to the stem mask information and the neighbor relationship between tomato ROIs and stem ROIs, the ROIs of pickable stems were determined. Meanwhile, the pickable stem edges were finely extracted from the stem ROI by using the thinning algorithm, together with expansion operation and shape characteristics of stem. Secondly, the picking point in image coordinate system was obtained, which was set as the center point of stem skeleton along the X (or Y) axis. Subsequently, the depth map of pickable stem ROI was used for obtaining the original depth value of picking point. Specifically, due to the large depth value errors, or even a lack of depth values when capturing small objects by the economical RGB-D depth camera. By using only the depth map corresponding to the stem mask area, the average depth value of picking point was calculated. The accurate depth value of picking point was obtained by comparing the average with the original depth value. Thirdly, according to the geometric features of fruit stem, the tangent slope of fruit stem at the picking point was calculated, and the search algorithm was used for finding the endpoints of fruit stem. Correspondingly, fruit stem direction was estimated by the vector composed of two endpoints of fruit stem. Finally, the picking point was converted to the robot coordinate system. Simultaneously, according to the tangent slope and the direction of fruit stem, the picking pose of the end effector was determined. Eventually, the robot was guided to complete the picking task with an appropriate pose. A large number of field test verified that the average recognition rate of pickings point was 98.07%, while the image resolution was 1280 pixel×720 pixel, the processing rate of the algorithm was 21f/s, the maximum positioning error of the image coordinates of picking points was 3 pixels, and the depth value error was ±4mm. After the picking points were successfully positioned, the picking rate was 98.15%. Compared with the existing similar methods, the positioning accuracy of the picking point was increased by 76.80 percentage points, the picking rate was increased by 15.17 percentage points, and picking efficiency was increased by 31.18 percentage points. Therefore, the proposed method fully met the requirements for robots in unstructured environment during harvesting.