A semiconductor memory device includes a plurality of memory blocks, a plurality of refresh block counters, a refresh word line counter, and an arbitration circuit. The plurality of refresh block counters generate block addresses of at least two memory blocks to select at least two memory blocks to be refreshed from the plurality of memory blocks. The refresh word line counter generates a common word line address that is common to the at least two memory blocks. The arbitration circuit generates at least one first word line address based on the at least two block addresses and the common word line address and arbitrate so that each word line indicated by the at least one first word line address is refreshed during a period in which a word line indicated by an externally applied second word line address is accessed.