A better basis for a further processing such as virtual view rendering, in form of a disparity map is achieved. To this end, the disparity map generation is done in two separate steps, namely the generation of two depth/disparity map estimates based on two different pairs of views of the scene in a manner independent from each other, with then comparing both depth/disparity map estimates so as to obtain a reliability measure for one or both of the depth/disparity map estimates.