A reference patch of an unknown environment is generated on the fly for positioning and tracking. The reference patch is generated using a captured image of a planar object with two perpendicular sets of parallel lines. The planar object is detected in the image and axes of the world coordinate system are defined using the vanishing points for the two sets of parallel lines. The camera rotation is recovered based on the defined axes, and the reference patch of at least a portion of the image of the planar object is generated using the recovered camera rotation. The reference patch can then be used for vision based detection and tracking. The planar object may be detected in the image as sets of parallel lines or as a rectangle.