An imaging system may include an image sensor having front side illuminated near infrared image sensor pixels. Each pixel may be formed in a graded epitaxial substrate layer such as a graded p-type epitaxial layer or a graded n-type epitaxial layer on a graded p-type epitaxial layer. Each pixel may be separated from an adjacent pixel by an isolation trench formed in the graded epitaxial layer. A deep p-well may be formed within each isolation trench. The isolation trenches and photodiodes for the pixels may be formed in the graded p-type epitaxial layer or the graded n-type epitaxial layer. The graded p-type epitaxial layer may have an increasing concentration of dopants that increases toward the backside of the image sensor. The graded n-type epitaxial layer may have an increasing concentration of dopants that increases toward the front side of the image sensor.