In order to reduce the misdetection ratio and false detection ratio of small casting defects in X radiographic testing，a dynamic defects tracking and detection method based on selective attention mechanism and deep learning feature matching was proposed. The misdetection of image sequences was eliminated with attention region detection of individual images and the false detection was also eliminated with feature matching among the image sequence. In the phase of individual images detection, a search method based on central-peripheral gradient was proposed to simulate the central-peripheral difference operation of biological vision. And the gradient threshold was defined. Then by comparing each regional gradient with threshold, the suspicious defect area was directly detected according to the gradient threshold. The defects did not need to be segmented from the suspicious image area. So the method avoided the great influence of the defects segmentation accuracy rate to defects tracking. In the phase of tracking among the image sequence，referencing to the deep learning hierarchy of human visual perception system, a deep learning network based on convolution neural was established. The gray level signal of the suspicious defect area was directly used as input. The network could automatically extract the essential feature which made up the deep learning feature vector. The similarity of feature vector was defined based on Euclidean distance. Defect tracking was achieved by similarity matching of suspicious defect regions in continuous frames. Then the noise and other false defects were eliminated. The experiments show that the false detection rate and the misdetection rate are less than 3%. The detection accuracy rate is more than 97%，which proved the method is advanced and effective.