Provided herein may be a semiconductor memory device and a method of operating the semiconductor memory device. The semiconductor memory device may include: a memory block including upper pages, a center page, and lower pages; a peripheral circuit configured to perform a read operation on the memory block; and a control logic configured to control the peripheral circuit to perform the read operation and control the peripheral circuit such that, during the read operation, based on a location of a selected page among the plurality of pages, a pass voltage to be applied to first adjacent pages disposed adjacent to the selected page in a first direction differs from a pass voltage to be applied to second adjacent pages disposed adjacent to the selected page in a second direction.