A method of forming a photo sensor includes the following steps. A substrate is provided, and a first electrode is formed on the substrate. A first silicon-rich dielectric layer is formed on the first electrode for sensing an infrared ray, wherein the first silicon-rich dielectric layer comprises a silicon-rich oxide layer, a silicon-rich nitride layer, or a silicon-rich oxynitride layer. A second silicon-rich dielectric layer is formed on the first silicon-rich dielectric layer for sensing visible light beams, wherein the second silicon-rich dielectric layer comprises a silicon-rich oxide layer, a silicon-rich nitride layer, or a silicon-rich oxynitride layer. A second electrode is formed on the second silicon-rich dielectric layer.