A semiconductor memory device includes an all bank select signal generation block configured to receive level signals including information on at least one bank which has been refreshed, and generate all bank select signals, in response to an all bank refresh command; and a bank block including a plurality of banks which are configured to be refreshed in response to the all bank select signals or are refreshed in response to per bank select signals which are enabled when the level signals are enabled.