An autonomous driving device includes an acquisition unit configured to acquire a value corresponding to a steering operation during an autonomous driving, and a control unit configured to perform the autonomous driving and to control switching of a driving state between the autonomous driving and a manual driving. In a case where a first value corresponding to the steering operation is equal to or greater than a first threshold value and a second value corresponding to the steering operation before the driving state is switched from the autonomous driving in progress to the manual driving is equal to or greater than a second threshold value, the control unit decelerates the vehicle and switches the driving state from the autonomous driving in progress to the manual driving.