A method of generating new views for a multi-view display system includes receiving a block-based disparity estimate between left image data and right image data for a frame of image data at a processor, upscaling the block-based disparity estimate to a pixel-based disparity for each pixel in the image date, performing motion vector based disparity interpolation and image interpolation simultaneously to produce left and right side interpolated data, and generating at least one new view using left and right side interpolated data.