A vehicle lamp control system includes a target irradiating direction setting unit which sets, in accordance with a running condition of a vehicle, a target irradiating direction of the lamp. An actuator controls an irradiating direction of the lamp to be directed in the target irradiating direction based on a reference direction. The actuator includes a reference setting section which executes a reference setting operation to set the irradiating direction of the lamp to be directed in the reference direction. The target irradiating direction setting unit includes a condition determining section which determines whether the reference setting section has executed the reference setting operation, and a reference setting command section which, when the condition determining section determines that the reference setting section has not yet executed the reference setting operation, instructs the reference setting section to execute the reference setting operation.