In a vehicle control device for an autonomous driving vehicle that autonomously travels based on an operation command, a gesture image of a person around the autonomous driving vehicle is acquired, and a stored reference gesture image is collated with the acquired gesture image. At this time, when it is discriminated that the gesture of the person around the autonomous driving vehicle is a gesture requesting the autonomous driving vehicle to stop, it is determined whether a disaster has occurred. When it is determined that the disaster has occurred, the autonomous driving vehicle is caused to stop around the person requesting the autonomous driving vehicle to stop.