Provided is an image pickup apparatus, including: a lens portion including a focus lens unit and a magnification-varying lens unit; an image pickup element movable in an optical axis direction; a moving unit for moving the image pickup element in the optical axis direction; a memory unit for storing, as a reference position, a position of the image pickup element serving as a reference in the optical axis direction, the position corresponding to a zoom position; an operation unit for operating a defocus amount; and a controller for controlling the moving unit to move the image pickup element based on the reference position and the defocus amount operated by the operation unit.