A restoration filter generation device according to one embodiment of the present invention includes: an information acquisition unit that acquires information showing a difference that depends on a color of an optical transfer function of an optical system; and a restoration filter generation unit that generates a restoration filter, which weakens restoration strength according to the difference that depends on the color of the optical transfer function on the basis of the information acquired by the information acquisition unit, and makes the restoration strength of the restoration filter weaker than the restoration strength of an ideal filter decided assuming that the difference that depends on the color of the optical transfer function does not exist. As a result, the overcorrection is reduced.