An image processing device obtains a depth information image representing a distribution of depth information about an object on the basis of first and second images that have different degrees of blurring and that are captured by an image sensor including groups each including pixels that obtain different color information about the object. The image processing device includes an extracting unit configured to extract signals, each corresponding to one of the pixels of each group, from the groups and generate first and second extracted images respectively corresponding to the first and second images, and a calculating unit configured to calculate the depth information image from a difference in degree of blurring between the first and second extracted images. The signals each corresponding to one of the pixels of each group and extracted from the groups are signals corresponding to pixels that obtain the same color information.