A semiconductor device includes a transistor, an isolation structure, and a fin sidewall structure. The transistor includes a fin extending from a substrate and an epitaxy structure grown on the fin. The isolation structure is above the substrate. The fin sidewall structure is above the isolation structure and is on a sidewall of the epitaxy structure. A method for manufacturing the semiconductor device is also disclosed.