One embodiment provides a system that prevents duplicate entries in a non-blocking TLB that supports multiple page sizes and speculative execution. During operation, after a request for translation of a virtual address misses in the non-blocking TLB, the system receives a TLB fill. Next, the system determines a page size associated with the TLB fill, and uses this page size to determine a set of bits in the virtual address that identify the virtual page associated with the TLB fill. The system then compares this set of bits with the corresponding bits of other virtual addresses associated with pending translation requests. If the system detects that a second virtual address for another pending translation request is also satisfied by the TLB fill, the system invalidates the duplicate translation request associated with the second virtual address.