An integrated circuit structure includes a first transistor, an interconnect structure, a first dielectric layer, polycrystalline plugs, a semiconductor structure and a second transistor. The first transistor is formed on a substrate. The interconnect structure is over the first transistor. The first dielectric layer is over the interconnect structure. The polycrystalline plugs extend from a top surface of the dielectric layer into the dielectric layer. The semiconductor structure is disposed over the first dielectric layer. The second transistor is formed on the semiconductor structure.