A method for fabricating semiconductor device is disclosed. The method includes the steps of: providing a substrate having a metal-oxide semiconductor (MOS) transistor thereon and a first interlayer dielectric (ILD) layer surrounding the MOS transistor; forming a source layer, a drain layer, a first opening between the source layer and the drain layer, and a second ILD layer on the MOS transistor and the first ILD layer, wherein the top surfaces of the source layer, the drain layer, and the second ILD layer are coplanar; forming a channel layer on the second ILD layer, the source layer, and the drain layer and into the first opening; and performing a first planarizing process to remove part of the channel layer so that the top surface of the channel layer is even with the top surfaces of the source layer and the drain layer.