A caching system, having a first instruction cache and second instruction cache, receives an instruction address from an instruction fetch unit. For an address resulting in a cache miss in both first cache and second cache, an instruction associated with the address is retrieved from computer memory and stored with the address in first cache. For an address resulting in a cache miss in second cache and a cache hit in first cache, the instruction associated with the address is retrieved from first cache. If no instruction in second cache matches the retrieved instruction, the retrieved instruction is stored in second cache and associated with the received address; otherwise the matching instruction is associated with the received address. For an address resulting in a cache hit in second cache, the instruction associated with the address is retrieved from second cache. The retrieved instruction is returned to the instruction fetch unit.