A system and method for automatically calibrating the aiming direction of a camera onboard a vehicle. The method uses GPS and solar almanac data to estimate the location of the sun relative to the vehicle, and compares this to the solar position in an image from the camera. The comparison is performed with an ongoing recursive calculation which yields an improved estimate of the sun position relative to the vehicle, while simultaneously calibrating the aiming direction of the camera. The aiming calibration data is used to improve the accuracy of the camera's images as used for all of the camera's other functions.