Certain example embodiments of this invention relate to vacuum insulating glass (VIG) units having improved seals made using two different frit-based edge seal materials, and/or methods of making the same. In certain example embodiments, a first frit material is applied around peripheral edges of first and second glass substrates. The first frit material, which may be bismuth-based in certain example embodiments, is fired with a heat treatment (e.g., thermal tempering) process. A second frit material, which may be VBZ-based in certain example embodiments, is applied and at least partially overlaps with the fired first frit material. The first frit material acts as a primer, and the second frit material helps seal together the VIG unit. The second frit material is fired at a significantly lower temperature that enables the glass to retain the temper or other strength imparted by the heat treatment.