An electronic device display may have a display cover layer. The cover layer may have a border that has an opaque masking material with an opening defining a light window for an ambient light sensor. The ambient light sensor may have a photodetector mounted in a light sensor housing. A molded clear plastic light diffuser may be used to diffuse light for the ambient light sensor that is passing through the light window. The light diffuser may reduce directionality in the ambient light sensor. The light diffuser may have an array of molded protrusions such as flat-topped cones. Alignment features may be formed in the light sensor housing and the light diffuser. Clips and other molded structures for attaching the light sensor to a mounting bracket or other structures may be molded into the light diffuser and light sensor housing.