The panoramic camera can obtain image information, within the scope of agricultural machinery around 360° coverage, with a large coverage and other characteristics, more conducive to agricultural machinery automatic navigation and obstacle avoidance. However, it is necessary to conduct image mosaic and fusion of the images acquired by the multilens as the test platform, so as to generate panoramic images for the research on obstacle avoidance of agricultural machinery. With a Lovol tractor for test platform, with a panoramic camera, images of the farmland in the experimental field were obtained, first of all, image preprocessing, mainly for cylindrical projection transformation unified coordinate system, after the SIFT algorithm based on feature points extraction of image feature points and matching for traditional SIFT algorithm matching errors and problems affecting the quality of image matching, using RANSAC algorithm, several optimization iteration to eliminate the effect of error matching points, in view of the image transformation matrix matching generated after, to prevent the instability of its linear results and further optimization results, the nonlinear LM algorithm was used to refine the image, and then the linear weighted smoothing algorithm was used to fuse the image to achieve the generation of panoramic image. Calculated by using the experimental image overlap coefficient of correlation effect of quantitative evaluation of image mosaicking, and 30 groups of image, a total of 60 images were processed by RANSAC algorithm and the LM algorithm, and the experimental results showed that after RANSAC algorithm processing, matching point obviously by mistake, the average geometric distance between the matching feature points offset was decreased significantly, fell by an average of 39.4013 pixels to 0.5819 pixels, the correlation coefficient was from 0.2878 to 0.7249, compared with the traditional method of removing mismatched points by manually setting the matching threshold of SIFT algorithm, the correlation coefficient of 0.7249 processed by RANSAC algorithm was obviously larger than that of 0.5933 when the threshold was set at 0.4 and 0.2007 when the threshold was set at 0.6, which proved that this study can be applied to image mosaic in many cases and eliminate mismatched points; after LM algorithm processing, the average geometric distance offset was further decreased from 0.5819 pixels to 0.5693 pixels, and the correlation coefficient was further increased from 0.7249 to 0.7261, proving that this study can further optimize the transformation matrix and improve the mosaic quality of panoramic images.