A system for sub-pixel disparity estimation is described herein. The system includes memory circuitry to store image data and at least one processor to execute instructions to calculate a first disparity for a set of reference views. The reference views correspond to a first subset of views among a plurality of sub-aperture views represented in the image data. The at least one processor is to refine the first disparity to a second disparity for the reference views. The second disparity has higher precision than the first disparity. The at least one processor is to map the second disparity from the reference views to a second subset of views among the plurality of sub-aperture views different than the first subset of views.