The present disclosure provides a semiconductor device. The semiconductor device includes a substrate having a gate region, source and drain (S/D) regions separated by the gate region and a first fin structure in a gate region in the N-FET region. The first fin structure is formed by a first semiconductor material layer as a lower portion, a semiconductor oxide layer as a middle portion and a second semiconductor material layer as an upper portion. The semiconductor device also includes a second fin structure in S/D regions in the N-FET region. The second fin structure is formed by the first semiconductor material layer as a lower portion and the semiconductor oxide layer as a first middle portion, the first semiconductor material layer as a second middle portion beside the first middle and the second semiconductor material layer as an upper portion.