A method of forming trench isolation with different depths of a semiconductor device is disclosed. A semiconductor substrate having a first mask layer formed thereon is first provided. A first etching process is performed with the first mask layer as an etching mask to form a shallow trench structure, followed by forming a first dielectric layer on the semiconductor substrate to fill the shallow trench structure. The first dielectric layer is then patterned to form a second mask layer which is used in a second etching process to form a deep trench structure. After that, a dielectric material is applied to fill the deep trench structure.