A method for stereo rectifying a pair of images include: for each image of the pair of images, determining a position of an epipole in an image plane associated with a first camera orientation of a camera that captures the image; for each image of the pair of images, positioning the epipole in a center of a virtual image plane associated with a second camera orientation of the camera; subsequent to the positioning, aligning the pair of images relative to each other by rotating around a stereo base line that intersects the epipoles of the pair of images, and rotating the virtual image planes to position the virtual image plane substantially parallel to the stereo base line and their normal vectors substantially parallel to each other so as to obtain a stereo rectified pair of image planes. An embodiment of the invention also relates to a method and system for accurately determining the epipoles when they are unknown.