Systems and methods for determining a location based on a direction sign. A method includes receiving a map of a physical area that includes a plurality of nodes connected by a links, where each link represents a travel path between connected nodes and has an associated distance value. The method includes receiving image data that indicates at least a first destination point of interest (POI) associated with a first physical direction and a second destination POI associated with a second physical direction. The method includes associating the first destination POI with a first node and associating the second destination POI with a second node. The method includes determining a location node of the plurality of the nodes according to the distances and directions of shortest paths to the first and second nodes, and returning a physical location corresponding to the determined location node.