A method for forming a semiconductor device includes the steps of providing a substrate having a memory region and a logic region, forming a memory stack structure on the memory region, forming a passivation layer covering a top surface and sidewalls of the memory stack structure, forming a first interlayer dielectric layer on the passivation layer, performing a post-polishing etching back process to remove a portion of the first interlayer dielectric layer and a portion of the passivation layer on the top surface of the memory stack structure, forming a second interlayer dielectric layer on the first interlayer dielectric layer and directly contacting the passivation layer, and forming an upper contact structure through the second interlayer dielectric layer and the passivation layer on the top surface of the memory stack structure to contact the memory stack structure.