A method, system, and computer program product are provided for prioritizing prefetch instructions. The method includes a processor issuing a prefetch instruction and fetching elements from a cache that can include a memory or a higher level cache. The processor stores the elements in temporary storage and monitors for accesses by an instruction. The processor stores a record representing the prefetch instruction. The processor updates the record with an indicator and issues a new prefetch instruction by comparing the new prefetch instruction to the record, based on the new prefetch instruction matching the prefetch instruction, assigning the indicator to the new prefetch instruction as a priority value, based on the new prefetch instruction not matching the prefetch instruction, assigning a default value to the new prefetch instruction as the priority value, and determining whether to execute the new prefetch instruction, based on the priority value of the new prefetch instruction.