The present disclosure provides a method of manufacturing a semiconductor device. The method includes: depositing a first semiconductor layer on an inner surface of a trench of a substrate; depositing a second semiconductor layer on the first semiconductor layer on the inner surface of the trench of the substrate, in which a dopant concentration of the first semiconductor layer is less than a dopant concentration of the second semiconductor layer; and depositing a third semiconductor layer on the second semiconductor layer to fill the trench of the substrate, in which a dopant concentration of the third semiconductor layer is less than the dopant concentration of the second semiconductor layer.