Methods described herein relate to reducing the computational intensity of vision-based localization. Methods may include: receiving sensor data from a vehicle traveling along a road; identifying one or more features of the environment from the sensor data; classifying the one or more identified features into one or more of a plurality of semantic classifications for the features; identifying map image data based on an identified location of the vehicle; identifying one or more features in the map image data; comparing one or more identified features of a first semantic classification with one or more features of the map image data of the first semantic classification; and registering a localized location of the vehicle within the environment based, at least in part, on the one or more identified features of the first semantic classification corresponding to the one or more features of the map image data of the first semantic classification.