An image pickup device includes an image pickup element that receives light of an object image formed by a picture-taking lens; and a focus detection filter section that is so disposed near the image pickup element and on the side of the picture-taking lens as to be able to be inserted or withdrawn. The focus detection filter section, from the side of the picture-taking lens, includes a field diaphragm that includes a plurality of openings; a condenser lens disposed near a focus detection area on the surface of the field diaphragm; a pupil division diaphragm that corresponds to the opening of the field diaphragm and includes a pair of openings arranged at such an interval as to secure focusing accuracy; and a re-imaging lens group that includes a plurality of re-imaging lenses that are so arranged as to correspond to the openings of each of the pupil division diaphragms. When the focus detection filter section is inserted into an picture-taking optical path, the light intensity distribution of two light beams that each pass through different areas of the picture-taking lens and the openings of the field diaphragm is received by the image pickup element. The detecting of focus is performed by detecting a phase difference of an output signal representing the light intensity distribution obtained from the image pickup element. When a picture is taken, the focus detection filter section withdraws from the optical path.