In one embodiment, in response to a first control command originated from a driver of an ADV, an expected acceleration of the ADV in response to the first control command is determined in view of a current speed of the ADV under the standard driving environment (e.g., dry road, flat road surface, normal tire pressure, zero load). One of the command calibration tables is selected based on a current driving environment of the ADV at the point in time. A lookup operation is performed in the selected command calibration table to obtain a second control command based on the current speed and expected acceleration of the ADV. The second control command is then issued to the ADV to control the ADV. As a result, the ADV would have reached the same acceleration under the current driving environment as if the ADV was driving in the standard driving environment.