A planar surface within a physical environment is detected enabling presentation of a graphical user interface overlaying the planar surface. Detection of planar surfaces may be performed, in one example, by obtaining a collection of three-dimensional surface points of a physical environment imaged via an optical sensor subsystem. A plurality of polygon sets of points are sampled within the collection. Each polygon set of points includes three or more localized points of the collection that defines a polygon. Each polygon is classified into one or more groups of polygons having a shared planar characteristic with each other polygon of that group. One or more planar surfaces within the collection are identified such that each planar surface is at least partially defined by a group of polygons containing at least a threshold number of polygons.