A vehicle control system to operate a vehicle autonomously with improved energy efficiency and ride quality is provided. A controller is configured to set a limit value of an operating point of an engine to reduce noises and vibrations. The operating point of the engine is restricted by the limit value within an acceptable region where an engine speed is higher than the limit value but an engine torque is lower than the limit value. The limit value is set to a first limit value when the vehicle is operated autonomously while carrying the passenger, and to a second limit value to expand the acceptable region when the vehicle is operated autonomously without carrying the passenger.