Traditional swine disease prevention primarily relies on human expertise, which risks missed diagnoses due to human error. To address this challenge, a multi-modal swine disease knowledge graph was developed to assist managers in better understanding the connections between pigs, providing a solid data foundation for identifying potential disease transmission paths and anomalies. Firstly, the swine disease data from various sources were collected, and then two preliminary multi-modal knowledge graphs were constructed through knowledge extraction and image matching. Secondly, a multi-modal knowledge graph fusion method based on “image-text” association was proposed, using a multi-head attention mechanism to reduce the impact of visual ambiguity and enhance swine disease entity representation. Finally, by calculating the similarity of entity representations in vector space, entities from the two multi-modal datasets were integrated into a more comprehensive knowledge graph. Experiments demonstrated that the proposed method improved alignment accuracy, as reflected by a 0.033 increase in Hits@1 compared with that of existing methods. Additional accuracy gains of 0.152, 0.236 and 0.180 were observed on the DBPZH-EN, DBPFR-EN, and DBPJA-EN datasets respectively, demonstrating its effectiveness in multi-modal knowledge graph fusion.