A machine vision based plug tray localization and detection system for automatic transplanting robot was proposed. This system could not only get the plug tray’s position on the conveyor belt, but also obtain the depth and the 3-D shape of the substrate inside each cave. Based on the registration of the color image and depth image, the contour of the plug tray was detected from the color image. Combined with the plug tray’s dimension, each cave was segmented from the depth image and the 3-D point cloud was generated. With K-nearest-neighbor (KNN) algorithm and principle component analysis (PCA) algorithm, the normal vector for each point was calculated. Based on the normal vector, the cave’s side wall was segmented from the substrate and the depth of the substrate was computed. The experiment result showed that this system could effectively localize and detect the plug tray with a localization error of 3.5 mm and a depth detection error of 4.9 mm which met the control requirement of the automatic transplanting robot for grafting seedlings. 