A pancake lens assembly includes a partially reflective mirror, a reflective polarizer, a quarter waveplate, a polarization-dependent optical device, and at least one polarization controller. When a light beam is introduced into the pancake lens assembly along an optical axis in a Z direction to pass through the polarization controller in a first state, a polarization direction of the light beam is converted by the polarization controller. When the light beam is introduced into the pancake lens assembly along the optical axis to pass through the polarization controller in a second state, the polarization direction of the light beam is prevented from being converted by the polarization controller.