An augmented reality system in which video imagery of a physical environment is combined with video images output by a game engine by the use of a traveling matte which identifies portions of the visible physical environment by techniques such as Computer vision or chroma keying and replaces them with the video images output by the video game engine. The composited imagery of the physical environment and the video game imagery is supplied to a trainee through a headmounted display screen. Additionally, peripheral vision is preserved either by providing complete binocular display to the limits of peripheral vision, or by providing a visual path to the peripheral vision which is matched in luminance to higher resolution augmented reality images provided by the binocular displays. A software/hardware element comprised of a server control station and a controller onboard the trainee performs the modeling, scenario generation, communications, tracking, and metric generation.