According to an image processing apparatus includes an acquisition unit, a storage unit, and a convolution unit. The acquisition unit is configured to acquire an input image captured via an optical system. The storage unit is configured to store therein a coefficient designed through learning for each of positions of respective pixel sets referred to in a convolution such that a result of the convolution of a second image with the coefficient is brought nearer to a first image, the second image being obtained by deteriorating the first image through a predetermined deterioration process. The convolution unit is configured to read the coefficient from the storage unit correspondingly to a position of a pixel set referred to in the input image, and generate an output image by convoluting the pixel set with the coefficient.