A gas turbine engine comprises a main compressor section having a high pressure compressor with a downstream discharge, and more upstream locations. A turbine section has a high pressure turbine. A tap taps air from at least one of the more upstream locations in the compressor section, passes the tapped air through a heat exchanger and then to a cooling compressor. The cooling compressor compresses air downstream of the heat exchanger, and delivers air into the high pressure turbine. The cooling compressor includes a downstream connection that delivers discharge pressure air to an upstream location in the high pressure turbine and a second tap from an intermediate pressure location within the cooling compressor. The second tap is connected to a downstream location within the high pressure turbine. An intercooling system for a gas turbine engine is also disclosed.