A method of fabricating a semiconductor device is provided. A stack layer is formed on a substrate. The stack layer is patterned to form a plurality of stack structures extending in a first direction. A trench extending in the first direction is located between two adjacent stack structures. Each trench has a plurality of wide portions and a plurality of narrow portions. A maximum width of the wide portions in a second direction is larger than a maximum width of the narrow portions in the second direction. A charge storage layer is formed to cover a bottom surface and sidewalls of the wide portion and fill up the narrow portion. A conductive layer is formed to fill up the wide portion. A semiconductor device formed by the method is also provided.