An image sensor having an array of pixels disposed in a substrate. Each pixel includes a photosensitive element, a color filter, and waveguide walls. The waveguide walls are disposed in the color filter and surround portions of the color filter to form waveguides through the color filter. The refractive index of the waveguide walls is less than the refractive index of the color filter. The image sensor may be back side illuminated (BSI) or front side illuminated (FSI). In some embodiments, metal walls may be coupled to the waveguide walls.