An image pickup unit that is capable of simplifying an optical adjustment work for an image pickup surface. A holding plate holds an image sensor. A lens barrel includes an optical system that forms an object image on the image sensor. A removable shim plate is inserted between the lens barrel and the holding plate and has a shape to avoid the image sensor. An energizing mechanism is inserted between the shim plate and the lens barrel to energizes the shim plate and the holding plate in a direction away from the lens barrel. Three adjusting screws fix the holding plate and shim plate to the lens barrel. Each of the adjusting screws displaces corresponding portions of the shim plate and holding plate toward the lens barrel when being fastened and displaces the corresponding portions away from the lens barrel in cooperation with the energizing mechanism when being loosened.