In polarimetric interferometric synthetic aperture ray (PolInSAR) forest structure parameter estimation, the data are affected by the baseline length size, signal-to-noise ratio, environmental topography, and radar wavelength, especially under complex forest environmental conditions, which can lead to errors in the observed complex coherence and thus affect the final inversion results. Firstly the effect of volume coherence selection on the RVoG three-stage forest canopy height inversion was explored, and the coherence farthest from the ground phase was selected as the volume coherence with the ground phase as the reference pixel by pixel. Secondly, the ground phase estimation method was improved by using two coherence linear fitting methods, Deming regression (DMR) and orthogonal regression (OGR), to improve the estimation of the ground phase, and different error ratios (0.3 and 0.6) were set in the DMR fitting method to compare the effects of the ground phase estimation method on the RVoG three-stage forest canopy height inversion. The results showed that the inversion accuracy of the inversion of volume coherence with ground phase as the reference pixel-by-pixel selection was improved compared with that of the complex coherence with the HV polarization channel directly. The coefficient of determination（R2）was increased from 0.349 to 0.383, and the mean square error（MSE） was decreased from 7.097m2 to 5.755m2. Based on the optimal selection of the volume coherence, the ground phase estimation method was improved by using Deming regression and orthogonal regression. It was shown that the least squares regression (LSR)-based ground phase estimation had the lowest accuracy of RVoG three-stage inversion, using DMR and OGR for coherence line fitting had a certain improvement in inversion accuracy compared with LSR, and the coefficient of determination (R2) of all inversion results was around 0.440, and all MSE was reduced by about 2m2. The conclusions indicated that the forest canopy height inversion using the RVoG three-phase method introduced some errors in the height inversion by using the traditional LSR estimation of the ground phase in the presence of errors in the complex coherence. Using other coherence linear fitting methods to overcome the influence of the complex coherence error can improve the final forest canopy height inversion results, and it was also more reasonable to choose volume coherence inversion method with the ground phase as the reference.