A cache memory circuit that evicts cache lines based on which cache lines are storing background data patterns is disclosed. The cache memory circuit can store multiple cache lines and, in response to receiving a request to store a new cache line, can select a particular one of previously stored cache lines. The selection may be performed based on data patterns included in the previously stored cache lines. The cache memory circuit can also perform accesses where the internal storage arrays are not activated in response to determining data in the location specified by the requested address is background data. In systems employing virtual addresses, a translation lookaside buffer can track the location of background data in the cache memory circuit.