In a vehicle display device including a HUD that displays a virtual image on a front field of a view of a driver and a viewpoint detecting camera that captures an image of the driver, a measuring camera is capable of capturing the virtual image. A calibration pattern is at a predetermined position with respect to a viewing angle of the virtual image and angles of view of the viewpoint detecting and measuring cameras. The calibration pattern is captured by the viewpoint detecting and measuring cameras, and the virtual image is captured by the measuring camera. The relative position and orientation between the viewpoint detecting camera and HUD are calculated as orientation information using an image captured by the viewpoint detecting camera and an image captured by the measuring camera. In addition, the position of the virtual image displayed by the HUD is adjusted using the calculated orientation information.