A high voltage (HV) semiconductor device includes: a semiconductor substrate having a first conductivity type; a gate structure disposed over a portion of the semiconductor substrate; a pair of spacers respectively disposed over a sidewall of the gate structure, wherein one of the spacers is a composite spacer comprising a first insulating spacer contacting the gate structure, a dummy gate structure, and a second insulating spacer; a first drift region disposed in a portion of the semiconductor, underlying a portion of the gate structure and one of the pair of spacers, having a second conductivity type opposite to the first conductivity type; and a pair of doping regions, respectively disposed in a portion of the semiconductor substrate on opposite sides of the gate structure, wherein the pair of doping regions include the second conductivity type and one of the doping regions is disposed in the first drift region.