There is provided an autonomous driving device that improves the convenience for a driver in the switching between autonomous driving and manual driving. When the driver discovers an obstacle which cannot be detected by an autonomous driving device, and performs a driving operation which causes a manual driving switching threshold value to be exceeded for an operation time less than or equal to an operation threshold value so as to avoid the obstacle, a control unit of the autonomous driving device switches autonomous driving to manual driving, and then the control unit switches manual driving to autonomous driving even if the driver does not perform an autonomous driving switching operation. For this reason, it is possible to improve the convenience for the driver in the switching between autonomous driving and manual driving.