An accurate identification of a continuously variable transmission (CVT) system can greatly contribute to the tractor power device and the control strategy, particularly to the energy saving and emission reduction for the power improvement. This study aims to improve the accuracy of the theoretical model, due to the variation of the characteristics for the common continuously variable speed system with the working conditions. Taking hydrostatic transmission (HST) and hydro-mechanical continuously variable transmission (HMCVT) as the research objects, the speed regulation and efficiency characteristics of the hydraulic transmission system were determined under the working conditions (including engine speed, output load torque, and displacement ratio). The full-factor test was adopted to comprehensively analyze the hydraulic transmission system characteristics. Among them, the engine speed, output load torque, and displacement ratio were set at the 3-, 11-, and 5-levels, respectively. The samples of hydraulic transmission system characteristics were obtained by the bench test (including the test sample data of speed regulation characteristics and efficiency characteristics). The test bench was mainly composed of the variable pump, constant motor, diesel engine, wet clutch, several groups of gear devices and transmission shafts, as well as the speed torque sensors and the load device. Before that, the basic performance of the test bench was tested by the variable pump displacement ratio adjustment test (Test 1) and wet clutch test (Test 2). The influence degree of working conditions was compared using the partial least squares (PLS) method. Furthermore, the parameter identification and model correction of the hydraulic transmission system characteristics were proposed to combine the original theoretical model with the improved simulated annealing (I-SA). The simulated annealing was used as the inner cycle to construct the outer cycle. The disturbance number of the simulated annealing was improved to introduce an adaptive variation function. The results show that the speed regulation characteristics were closely related to the displacement ratio, depending mainly on the load torque, according to the bench test data from the hydraulic transmission system. PLS analysis showed that the influence degrees of the engine speed and output load torque (absolute value) were 0.36 and 0.92, respectively. The revised characteristics model of hydraulic transmission system speed regulation was the optimal combination of the original theoretical model and the first-order linear model of load torque. The accuracy of the revised model was significantly improved (34.41%) than before. The efficiency characteristics were mainly related to the displacement ratio and load torque, according to the bench test data of efficiency characteristics for the hydraulic transmission system. Among them, the influence degrees (absolute value) of the engine speed, output load torque, and displacement ratio were 0.05, 0.71, and 0.26, respectively. There was the limited accuracy of the original 6-parameter theoretical model (the mean absolute percentage error about 4.76%, and R2 about 0.70) after parameter identification, indicating the different overall change from the actual measurement. By contrast, the new semi-empirical modified model can be expected to effectively modify the original theoretical model. The mean absolute percentage error of the newly-developed 5-parameter semi-empirical modified model was improved by 47.48% than before, where the R2 was 0.93 (improved by 32.86%). The characterization of the new model was highly consistent with the actual measured values. Specifically, there was a large value in the conditions of large displacement and large load, indicating a divergent decline in the conditions of medium or small displacement and load, i.e., the gradually increased decline. Therefore, the I-SA algorithm can be expected to effectively serve as the engineering practice by introducing the outer cycle and the adaptive change of disturbance number. The reasonable design and control strategy can then be achieved in the correct speed regulation and efficiency characteristics model for the better performance of the tractor CVT system.