Methods, systems and apparatus, including computer program products, for generating a sitemap where a last modified time of website content is not available to a network crawler. A sitemap generator determines that a file has a previously stored version number, compares a current version number of the file to the previously stored version number of the file, generates a sitemap having an entry corresponding to the file if the current version number of the file is changed from the previously stored version number, and assigns a new last modified time to a last modified time value for the entry wherein an actual last modified time of the file is not available. A search engine receives the sitemap, determines that the entry was previously indexed at a time before the last modified time value, and accesses file information associated with the file to update an index with the file information.