An image pickup unit, an image pickup device, a picture processing method, a diaphragm control method, and program are capable of suppressing deterioration in quality of a stereoscopic picture. A parallax detection pixel receives object light by a plurality of photodetectors covered with one microlens, to generate a signal used for detecting parallax. G pixels, an R pixel, and a B pixel each receive the object light to generate a signal used for generating a planar picture. A parallax detection section detects parallax based on the signal generated by the parallax detection pixels. A 2D picture generation section generates a planar picture based on a signal generated by picture generation pixels. A 3D picture generation section adjusts a position of each object image included in the planar picture, based on the detected parallax, to generate a stereoscopic picture.