A vehicle driving control system has an automatic driving mode that enables traveling without a driver's operation. The vehicle driving control system includes a driver monitor that monitors a driver in a vehicle compartment, and detects a driver state, a driver state determining unit that determines, based on the driver state, whether the driver is in a state capable of driving normally, a retreat mode controller that controls a retreat mode that causes an own vehicle to travel for retreat to a place where the own vehicle can safely stop when it is determined, during traveling under the automatic driving mode, that the driver is not in the state capable of driving normally, and a retreat mode canceler that cancels the retreat mode when it is determined, during the travel for retreat, that the driver has returned to the state capable of driving normally.