The Fe, Co and Cu modified HZSM-5 zeolites were prepared via impregnation method and the modified catalysts were characterized by XRD, Py-IR and BET methods. Then online catalytic upgrading of rape straw pyrolysis vapors was performed over the modified catalysts to investigate their catalytic upgrading performance and anti coking performance. The results showed that loadings of Fe, Co and Cu were well-distributed, the acid distributions on the HZSM-5 were influenced differently by Fe, Co and Cu, and the channels were modified with the decrease of pore volume. The bio-oil yields were decreased with the increase of physicochemical properties after upgrading by employing the modified HZSM-5. The yield, oxygen content, pH value, dynamic viscosity and high heating value (HHV) of biooil obtained by using Fe, Co and Cu/HZSM-5 were 18.37%~19.03%, 15.13%~17.23%, 5.05~5.12, 5.16~5.22mm2/s and 34.56~36.01 MJ/kg, respectively. The refined bio-oil was composed of a variety of organic compounds which was indicated by many kinds of chemical functional groups, the content of hydrocarbons in refined bio-oil was increased significantly. Aromatization performances of Fe/HZSM-5, Co/HZSM-5 and Zn/HZSM-5 were improved and contents of the polycyclic aromatic hydrocarbons (PAHs) were obviously increased. And after catalytic upgrading by Fe/HZSM-5 and Co/HZSM-5, the contents of the monocyclic aromatic hydrocarbons (MAHs) were relatively high. Resistance performances of Fe/HZSM-5 and Co/HZSM-5 for amorphous coke were relatively strong, while that of Co/HZSM-5 for graphite coke was enhanced obviously.