A system and method for reconstructing a three-dimensional (3D) model are described. The 3D model reconstruction system includes: a low resolution reconstruction unit that converts a first depth map, acquired by scanning a scene with a depth camera, into a second depth map having a low resolution, processes the second depth map to extract pose change information about a pose change of the depth camera, and reconstructs a low resolution 3D model in real-time; and a high resolution reconstruction unit that processes the first depth map by using the pose change information of the depth camera that is extracted from the second depth map and reconstructs a high resolution 3D model.