A method of controlling autonomous driving is provided. The method includes collecting driving information of a driver and curvature information of a road and generating a driving pattern of the driver, defined by associating behaviors in longitudinal and lateral directions of the vehicle based on the driving information. The driving pattern is then set to a constraint condition for driving torque and brake pressure and the vehicle is operated based thereon.