A method includes performing a hybrid bonding to bond a first package component to a second package component, so that a bonded pair is formed. In the bonded pair, first metal pads in the first package component are bonded to second metal pads in the second package component, and a first surface dielectric layer at a surface of the first package component is bonded to a second surface dielectric layer at a surface of the second package component. After the hybrid bonding, a thermal compressive annealing is performed on the bonded pair.