A control mode of speed control includes a first control mode in which an actual speed is caused to follow a target speed through active use of a brake and a second control mode in which the actual speed is caused to follow the target speed while use of the brake is suppressed. An autonomous driving system changes a threshold value to a value larger than a value in the first control mode when the control mode is switched from the first control mode to the second control mode, and changes the threshold value to a value smaller than a value in the second control mode when the control mode is switched from the second control mode to the first control mode.