A wavelength selective optical switch includes a light input/output unit having a plurality of input/output ports, a polarization plane-independent wavelength dispersion element that splits incident light input from the optical input/output unit into spatially different angles for each wavelength, and synthesizes emergent light from different directions and outputs the light to the optical input/output unit, a condenser element that condenses the light split by the wavelength splitting element, a polarization splitter that splits incident light incident via the condenser element according to a polarization component to result in first and second light beams, aligns a polarization direction by rotating a polarization direction of one of the beams, and synthesizes the incident light by rotating one polarization direction of the emergent light of the same wavelength among the first and second reflected light beams, and a space phase modulation element.