An autonomous driving control system for a vehicle which is able to switch between manual driving and autonomous driving is provided with a driver condition sensor, acting part, and electronic control unit. The electronic control unit is provided with an autonomous driving control part, reliance calculating part for calculating an autonomous driving output reliance, vigilance calculating part for calculating a driver vigilance, and an action control part for controlling a strength of an action against a driver. In a region in which an operating point determined by the autonomous driving output reliance and driver vigilance can fall, a plurality of sub regions are defined by boundary lines extending so that the driver vigilance becomes higher as the autonomous driving output reliance becomes lower. The action control part controls the strength of the action against the driver to differ in accordance with the sub region in which the operating point falls.