A first image and a second image of a scene are obtained for use in detecting the location of the floor in the scene. A first of the images is mapped onto a candidate plane that is perpendicular to a gravity vector, thereby creating a texture-mapped plane. The texture-mapped plane is projected into the geometry of the camera utilized to take the second image to create an artificial first image. A comparison is then performed between the first image and the artificial first image to generate a similarity score. Multiple candidate planes are processed in this manner for the first and second images, and the candidate plane generating the highest associated similarity score is chosen as the floor. This process can be repeated for multiple sets of stereo images.