A semiconductor device and method of forming the same are provided. The semiconductor device includes a first conductive line over a substrate and a memory structure over the first conductive line. The memory structure is electrically coupled to the first conductive line through a conductive via. A spacer layer is laterally aside the memory structure and covers sidewalls of the memory structure. A first dielectric layer is on the spacer layer and laterally aside the memory structure. A second dielectric layer is on the memory structure, the spacer layer and the first dielectric layer. A second conductive line penetrates through the second dielectric layer, the first dielectric layer and the spacer layer to electrically couple to the memory structure. The second conductive line includes a body part at least partially embedded in the second dielectric layer and an extension part underlying the body part and laterally protruding from a sidewall of the body part. The extension part is electrically connected to an upper electrode of the memory structure and surrounded by the spacer layer.