A system and method that generates a planogram from a realogram is disclosed. The method includes receiving, from an image processing module, a realogram, the realogram including an image and information about a set of items recognized in the image, generating a bounding box containing the set of recognized items, identifying a subset of the set of recognized items corresponding to a shelf, determining a location of each gap of a number of gaps on the shelf, determining a physical width of each gap of the number of gaps, and generating, from the realogram, a planogram based on the bounding box, the set of recognized items, the location, and the physical width of each of the number of gaps.