A vehicle management system for managing vehicles used by users has a usage managing unit configured to accept a user's request for using to use one of the vehicles, and manage the vehicles and parking spaces in which the vehicles are parked, an acquisition unit configured to acquire energy information from the vehicles, the energy information indicating a remaining amount of energy used for traveling of the vehicle, a determination unit configured to compare the remaining amount acquired by the acquisition unit with a remaining amount threshold that represents a shortage of energy, determine that the vehicle needs supply of energy when the remaining amount is lower than the remaining amount threshold, and specify a recovery vehicle that has been determined as needing supply of energy as a recovery vehicle, and a command outputting unit configured to output a recovery command for recovering the recovery vehicle from the parking space.