In certain example embodiments of this invention, a window unit may include a vacuum IG (VIG) unit as an inboard lite and a monolithic lite (e.g., with an optional low-E coating thereon) as an outboard lite. A dead air space may separate the inboard and outboard lites. A highly insulated frame may be used to support the inner and outer lites. The VIG unit may be partially embedded or supported in the insulative frame, so that the insulating frame separates the VIG unit inboard lite from the outboard lite thereby reducing conductivity around the edges of the window unit so that R-value can be increased (and U-value decreased). In certain example embodiments, the total R-value of the window unit is at least about R-8, and more preferably at least about R-10 (compared to the much lower R-values of conventional IG units).