Rice cracking occurring in the hot air drying of rice kernels reduced its quality and economic values. The reason for the rice cracking is due to the local drying stress of the kernel exceeds its resistance. The drying stress includes the stresses induced by temperature and water gradient, so the temperature and water gradient is basis for predicting drying stress. The purpose was to predict the heat and mass transfer process inside the rice kernel. The image processing technology was applied to construct a three-dimensional body-fitted grid for the rice kernel. The governing equations and relative boundary conditions were developed for the heat and mass transfer processes respectively. New measuring technology (TPS) was used to measure the properties of rice kernel such as the heat conduction and mass diffusion coefficient. Finally, the Comsol Multiphysics software was applied to solve the mathematical model. The simulation results were compared with the experimental data and the difference was less than 8%, indicating high model accuracy. The temperature and water distribution gradients in the rice kernel along the radial direction (short axis) were larger than those along the axial direction (long axis), and the duration time of water gradient existence was longer than temperature gradient existence. The moisture gradient in the radial direction from the outer surface to the one-third of center was larger than that in other parts of the radial direction, which was consistent with the cracking phenomena observed in experiments—the crack extended from the surface inward. The image processing technology was applied to construct three-dimensional body-fitted grid for the rice kernel. New measuring technology (TPS) was used to measure the properties of rice kernel.