A method of manufacturing a polarization-modulating optical element is provided. The element causes, for light passing through the element and due to stress-induced birefringence, a distribution of retardation between orthogonal states of polarization. The method includes joining a first component and a second component. A non-plane surface of the first component is provided with a defined height profile is joined with a plane surface of the second component. A mechanical stress causing the stress-induced birefringence is produced in the such formed polarization-modulating optical element.