An algorithm based on image processing technology was proposed for generating navigation directrix in orchard of complex circumstance. Image of R—B component was binarized by 2-D maximum between-cluster variance. A region of interest was detected by horizontal projection method to extract tree trunks. Crossing points between tree and ground were obtained by scanning the trunks and classified into different clusters in terms of their neighboring relation. Least square method was used for detecting two border lines, center points of crossing points between two border lines were extracted to generate navigation directrix for agricultural robot. The experimental results under various scene conditions showed that the algorithm could generate navigation directrix accurately, the accuracy was more than 90.7% which showed a good noise robust；the average processing time of a 640×480 pixels image was less than 119ms and met the need of real-time.