An insulation window for use with a water heater having a tank and a jacket together at least partially defining a circumferential void therebetween, the insulation window comprising: (a) a rigid polymer core including a continuous wall defining an internal through hole; (b) a first compressible layer overlapping a first side of the continuous wall to delineate a first opening in communication with the through hole; and, (c) a second compressible layer overlapping a second side of the continuous wall to delineate a second opening in communication with the through hole, where the first side of the continuous wall is generally opposite the second side.