Support vector regression (SVR) is an effective tool for machine error modeling. To improve the predicted performance of SVR model, the core vector regression (CVR) algorithm which is suitable for resolving the training of large-scale sample data was introduced into thermal error modeling for CNC machine tool. Principal components were firstly extracted from the sample set using the feature extraction of partial least squares (PLS) algorithm to construct the feature set, which would reduce the number of state variables without information loss by dimension reduction, data de-noising and eliminating the correlative between variables. Then improved particle swam optimization (IPSO) was applied for determining the parameters of CVR to get the optimal performance of the thermal error model, and the proposed combined method was called PLS—IPSO—CVR. Experimental results showed that the training speed of PLS—IPSO—CVR model was much faster and it produced fewer support vectors on very large sample data in comparison with SVR and BP neural network. Thus the feasibility and effectiveness of this combined modeling method was verified.