In one aspect, a method of extracting depth information from a stereoscopic pair of images comprises: in a first depth extraction phase, determining a set of support points in 3D space for at least a first image of the stereoscopic image pair; using the set of support points to determine a 3D surface prior; and in a second depth extraction phase, computing, for the first image, a second-stage disparity map, by optimizing a second cost function defined so as to penalize deviation in the second-stage disparity map from the 3D surface prior. Disparity smoothness may be encouraged in at least one of the first and second depth extraction phases using a cost function that penalizes deviations in disparity between nearby pixels.