When a main block address held in a memory refresh address counter coincides with an access block address corresponding to an access request, its counter value is transferred to a sub refresh address counter. Thereafter, a sub refresh address counter operates with priority over a main refresh address counter until its counter value reaches a final value. Consequently, an access operation and a refresh operation can be simultaneously executed without interfering with each other. As a result, it is possible to execute the refresh operation with a minimum increase in circuit scale and without any deterioration in access efficiency.