Electronic devices may be provided with image sensors. Image sensors may be configured to capture images during imaging operations and monitor ambient light levels during non-imaging operations. An image sensor may include image pixels that receive light and dark pixels that are prevented from receiving light. An image sensor may include an ambient light detection circuit. The ambient light detection circuit may include an oscillator, timing and control circuitry, and a counter. The oscillator may be switchably coupled to the image pixels and the dark pixels. The counter may be configured to count up oscillator cycles of the oscillator while the oscillator is coupled to the image pixels and to count down oscillator cycles of the oscillator while the oscillator is coupled to the dark pixels. The counter may provide a count value that depends on a signal from the image pixels and a signal from the dark pixels.