A semiconductor memory device configured to perform a clock synchronous burst read operation includes a plurality of buffer memories having different bank structures, and first and second data latch circuits storing read data read from the plurality of buffer memories. The semiconductor memory device further includes a control circuit that controls a timing of starting counting up addresses and a timing of storing read data in the first data latch circuit at the time of the clock synchronous burst read operation in accordance with the bank structure of the buffer memory as a read operation target.