In image processing device, a basic coordinate system corresponding to arrangement directions of the pixel in the two-dimensional image sensor is transformed to a virtual coordinate system corresponding to an arrangement rule of the filters of each of the wavelength components, position calculation processing for image processing containing an aberration correction in the virtual coordinate system corresponding to each of the wavelength components is executed to determine coordinates in the virtual coordinate system, and the coordinates in the virtual coordinate system obtained by the position calculating unit are transformed to coordinates in the basic coordinate system.