A real-time perception adjustment and correction and driving adaption method for autonomous driving is provided. The system will analyze the driving behaviors of surrounding vehicles surrounding an ADV, which is utilized to improve its original perception based on the analysis and to adapt to the updated driving environment. In one embodiment, in addition to the perception information provided by the sensors, the system analyzes the behaviors of the surrounding vehicles based on the perception information. Based on the behaviors of the surrounding vehicles, the system may detect there is may be an obstacle that has not been detected based on the perception information. Alternatively, the system may detect that an obstacle determined based on the perception information actually may not exist based on the behaviors of the surrounding vehicles. The paths created for the ADV may then adjusted accordingly to improve the autonomous driving of the ADV.