Discrimination of ripe asparagus and accurate location of the picking hand is a challenge in the selective harvesting process of asparagus harvesting robots. To address this challenge, an improved you only look at coefficients (YOLACT++) based algorithm was proposed, which was used to detect and discriminate ripe asparagus and locate harvesting cuts. Improving the traditional YOLACT++ backbone feature extraction network, specifically including the introduction of a convolutional block attention module (CBAM) attention mechanism and a spatial pyramid pooling (SPP) module, to improve the effectiveness of the network for feature extraction and enhance its detection segmentation results. Asparagus have different sizes and postures, by designing different anchor frame sizes to ensure that they were covered, the adaptability of the anchor frame to the aspect ratio of the asparagus was improved, thus improving the detection accuracy and speed of the network. The skeleton was then fitted to asparagus with varying growth forms. Determination of asparagus maturity after calculating asparagus length and basal diameter in segments. Finally, the location of the cutting point in the bottom area of the mature asparagus was calculated, and its spatial location was determined by quantifying the roll angle and pitch angle to locate the final harvesting cutting surface. The results of the harvesting robot field trials showed that the detection accuracy of the trained improved YOLACT++ model was 95.22%, the average accuracy of the mask was 95.60%, the detection time of 640 pixels×480 pixels size image was 53.65ms, the accuracy of mature asparagus discrimination was 95.24%, the error of cutting point positioning in X, Y and Z directions was less than 2.89m, and the maximum error in rotation and pitch angles was 7.17°. Compared with that of the Mask R-CNN, SOLO and YOLACT++ models, the average accuracy of the mask was improved by 2.28, 9.33 and 21.41 percentage points, respectively;the maximum positioning errors were reduced by 1.07mm, 1.41mm and 1.92 mm, respectively, and the maximum angle errors were reduced by 1.81°, 2.46° and 3.81°, respectively. The harvesting success rate of the trial asparagus harvesting robot was 96.15%, and that the total time taken to harvest a single asparagus was only 12.15s. The detection-discrimination-location method proposed had high detection and location accuracy, which ensured detection speed on the premise. It can provide technical support for optimizing and improving the asparagus harvesting robot based on machine vision.