A method includes bonding a first device die with a second device die. The second device die is over the first device die. A passive device is formed in a combined structure including the first and the second device dies. The passive device includes a first and a second end. A gap-filling material is formed over the first device die, with the gap-filling material including portions on opposite sides of the second device die. The method further includes performing a planarization to reveal the second device die, with a remaining portion of the gap-filling material forming an isolation region, forming a first and a second through-vias penetrating through the isolation region to electrically couple to the first device die, and forming a first and a second electrical connectors electrically coupling to the first end and the second end of the passive device.