A semiconductor image sensor device includes a semiconductor substrate, a radiation-sensing region, and a first isolation structure. The radiation-sensing region is in the semiconductor substrate. The first isolation structure is in the semiconductor substrate and adjacent to the radiation-sensing region. The first isolation structure includes a bottom isolation portion in the semiconductor substrate, an upper isolation portion in the semiconductor substrate, and a diffusion barrier layer surrounding a sidewall of the upper isolation portion.