An image pickup apparatus capable of obtaining a smooth video image and of reducing deterioration in the vertical resolution and an image stabilization performance. An image pickup device has a valid pixel area for obtaining an image signal, an optical black area for an optical black correction, and an amplifier unit that amplifies an output from the pixel area and outputs an image signal. A gain adjustment unit adjusts gain of the image signal. A reading control unit changes a gain value in the amplifier unit, changes the numbers of lines in the areas, and performs reading control of the image pickup device, when a control gain value in the gain adjustment unit exceeds a reference value. A change unit changes a time constant of a correction filter used for the optical black correction when the control gain value exceeds the specified reference value.