An illumination system of a microlithographic projection exposure apparatus includes a spatial light modulator which varies an intensity distribution in a pupil surface. The modulator includes an array of mirrors that reflect impinging projection light into directions that depend on control signals applied to the mirrors. A prism, which directs the projection light towards the spatial light modulator, has a double pass surface on which the projection light impinges twice, namely a first time when leaving the prism and before it is reflected by the mirrors, and a second time when entering the prism and after it has been reflected by the mirrors. A pupil perturbation suppressing mechanism is provided that reduces reflections of projection light when it impinges the first time on the double pass surface, and/or prevents that light portions being a result of such reflections contribute to the intensity distribution in the pupil surface.