Provided are a computer program product, system, and method for using an access increment number to control a duration during which tracks remain in cache. Tracks in a storage in the cache are indicated in a cache list. For each of the tracks indicated in the cache list, an access value is updated when one of the tracks is accessed in the cache. An access to a track in the cache indicated in the cache list is received. A determination is made as to whether an access increment number for the accessed track, wherein the access increment number is greater than one. The access value for the accessed track is incremented by the determined access increment number in response to the track being accessed in the cache. The access value for one of the tracks is used to determine whether to initiate to demote the track from the cache.