The method of remote sensing extraction of abandoned land is mainly classified into image classification and change detection. It is easy to be mixed with grassland and shrub because of the complexity of abandoned land cover types, resulting in the classification method of extraction accuracy is not high. The change detection method is susceptible to the interference of non-cultivated land change factors, and it can only extract new abandonment within the scope of data coverage, but it cannot extract historical abandonment before remote sensing data. In addition, remote sensing data have their limitations, the extracting ability of low and medium-resolution data is insufficient due to the interference of mixed pixels, and the precision of high-resolution remote sensing is vulnerable to the interference of topographic fluctuation, cloud cover and long coverage period. Therefore, it is difficult to extract abandoned land by traditional remote sensing method. To solve the above problems, a joint detection method of multiple source data was proposed to extract abandoned land. Based on the heterogeneity of multi-source data and the complementarity of different methods, different extraction strategies were formulated for different types of abandoned land, and coupling analysis was carried out to extract abandoned land. The field investigation showed that the total accuracy of the method was 97.6%. In addition, data mining for multi-source data and detection results can extract physical and geographical indicators such as “distance feature”, “height difference feature”, “irrigation feature” and “neighborhood feature”, and the significance analysis was helpful to distinguish the dominant factors of abandonment, which provided basis for the study of abandonment driving forces and the directional promotion of abandonment management methods.