A head-mounted display may include a display system and an optical system in a housing. The display system may have a pixel array that produces light associated with images. The display system may also have a linear polarizer through which light from the pixel array passes and a quarter wave plate through which the light passes after passing through the quarter wave plate. The optical system may be a catadioptric optical system having one or more lens elements. The lens elements may include a plano-convex lens and a plano-concave lens. A partially reflective mirror may be formed on a convex surface of the plano-convex lens. A reflective polarizer may be formed on the planar surface of the plano-convex lens or the concave surface of the plano-concave lens. An additional quarter wave plate may be located between the reflective polarizer and the partially reflective mirror.