Aiming at the problem that the current manual feature detection of assembly robots was susceptible to interference factors such as illumination conditions, background and occlusion, and the feature detection based on point cloud depends on the accuracy of model construction, the method of deep learning was proposed to carry out research on the visual positioning technology of the workpiece based on key point prediction. Firstly, the ArUco pose detection marker and ICP point cloud registration technology were used to construct a set of data for training the pose estimation network model. The depth images from various angles of the workpiece were collected, and the pose information of the workpiece was calculated. The key points on the workpiece surface were selected as the data set. Then the vector field of the key points on the workpiece surface was constructed, and the depth training was carried out to gather with the data set to realize the vector field prediction of the foreground points pointing to the key points. And the direction vectors of each pixel in the vector field pointing to the same key point were divided into two groups, the intersection points of their vectors were taken to generate the hypothesis of the key point, and all the hypotheses were evaluated based on RANSAC voting. The EPnP solver was used to calculate the pose of the workpiece, and the orientation bounding box of the workpiece was generated to display the pose estimation results. Finally, the accuracy and robustness of the estimation results were verified by experiments.