A method of tomatoes segmentation based on RGB-D depth images and K-means optimized SOM neural network was proposed, aiming to solve the problem of automatic recognizing and localizing difficulties caused by fruits overlapping and adherence. Firstly, the contours information of the fruits was obtained from preprocessed images taken by an RGB-D camera. Secondly, two-dimensional information and depth information of the points of contours were filtered and processed. Thirdly, the processed information was used as the input to the SOM neural network optimized by the K-means algorithm for training and a model for the point cloud clustering was established. Finally, the position and contour shape of each tomato were obtained. To verify the performance of the algorithm, the correct rate and the root mean square error of the fruit recognition results was used as evaluation indicators. Totally 80 pictures containing 366 tomatoes were taken as the sample, and accuracy, precision, sensitivity and specificity were taken as evaluation indicators. The correct rate was 87.2%, the root mean square error was 1.66mm. It was proved that the method had higher accuracy and better robustness compared with the method for two-dimensional images based on Hough transform. This method solved the problem of occlusion of tomato fruits in real environment to a certain extent, and provided a new idea for combining the three-dimensional coordinate information and self-organizing neural network for fruit segmentation.