An image sensor may include an array of imaging pixels and an array of color filter elements that covers the array of imaging pixels. The array of imaging pixels may include visible light pixels that are covered by visible light color filter elements and near-infrared light pixels that are covered by near-infrared light color filter elements. The imaging pixels may be arranged in a pattern having a repeating 2×2 unit cell of pixel groups. Each pixel group may include a visible light pixel sub-group and a near-infrared light pixel sub-group. Signals from each pixel group may be processed to determine a representative value for each pixel group that includes both visible light and near-infrared light information.