A system and method of capturing and rendering stereoscopic panoramic content is provided. The system may capture information from a virtual scene to generate a stereoscopic panorama in the form of a stereoscopic cube map. Using the pre-captured cube map, the system may render stereoscopic images during gameplay, image viewing, and/or playback of a three dimensional video. The system may capture camera distance information during capture for use in distortion reduction during image rendering and playback. The system may implement additional distortion reduction measures. Further, the system may use the capture camera distance information to write to depth buffers corresponding to rendered images for use when introducing dynamic gameplay elements into the scene. Additionally, the system may use realistic camera models to simulate human anatomy and reduce distortions associated with simplistic camera models.