The subject matter described in this specification is directed to a computer system and techniques for determining a location of an autonomous vehicle. The computer system is configured to determine the location using localization data from multiple data sources. When the localization data from the multiple data sources is unavailable or inaccurate, the computer system is configured to determine the location using predefined features of the environment.