A capacitor structure includes an insulative layer, a first electrode over the insulative layer, a dielectric layer over the first electrode, and a second electrode over the dielectric layer. The first electrode includes a first portion extending along a lateral direction of the insulative layer and a second portion connected to the first portion and extending along a depth direction of the insulative layer. The dielectric layer is substantially conformal with respect to a profile of the first electrode. A semiconductor structure thereof and a method for forming the same are also provided.