In an embodiment, a processor includes at least one core and a first cache memory including a first plurality of sets having a first plurality of cache lines and associated metadata to store address information, recency information and a first indicator to indicate whether the cache line is associated with an oversubscribed set of a second cache memory. A first cache controller may be configured to base an eviction decision with regard to a first set of the first plurality of sets including a first cache line at least in part on the first indicator of the first cache line. Other embodiments are described and claimed.