An integrated circuit structure includes a semiconductor substrate; a diode; and a phase change element over and electrically connected to the diode. The diode includes a first doped semiconductor region of a first conductivity type, wherein the first doped semiconductor region is embedded in the semiconductor substrate; and a second doped semiconductor region over and adjoining the first doped semiconductor region, wherein the second doped semiconductor region is of a second conductivity type opposite the first conductivity type.