The uniformity dynamic of moisture content of litchi fruit during infrared drying was measured and fitting analyzed. Besides, the dynamic differences of the shell porosity and its fractal dimension of litchi during infrared drying were compared using Image-Pro and SPSS software. Furthermore, the change of the color of shell surface color of litchi during fractal dimension was also measured. The results show that the moisture content change of each litchi fruit is uneven during infrared drying. For relatively small litchi, the moisture content decreases quickly during initial drying; later on, the moisture content changes slowly, and during the last drying the difference among individual drying rates becomes significant. The fruit shell porosity significantly changes with the reduction of moisture content, and its mean value decreases from 0.519 to 0.381, to 0.276, and then to 0.184; but the fractal dimension of the pore increases from 1.486 to 1.674, to 1.708, and then to 1.800. The DL and Db values of shell surface color change significantly at the beginning, however, they change slowly during the following drying. Results from this study could provide a reference for the uniformity research of other multilayer-structure fruits and vegetables in infrared drying.