A method includes bonding a first and a second device die to a third device die, forming a plurality of gap-filling layers extending between the first and the second device dies, and performing a first etching process to etch a first dielectric layer in the plurality of gap-filling layers to form an opening. A first etch stop layer in the plurality of gap-filling layers is used to stop the first etching process. The opening is then extended through the first etch stop layer. A second etching process is performed to extend the opening through a second dielectric layer underlying the first etch stop layer. The second etching process stops on a second etch stop layer in the plurality of gap-filling layers. The method further includes extending the opening through the second etch stop layer, and filling the opening with a conductive material to form a through-via.