One illustrative method disclosed herein includes removing a portion of a sacrificial sidewall spacer to thereby expose at least a portion of the sidewalls of a sacrificial gate electrode and forming a liner layer on the exposed sidewalls of the sacrificial gate electrode. In this example, the method also includes forming a sacrificial gap fill material above the liner layer, exposing and removing the sacrificial gate electrode to thereby define a gate cavity that is laterally defined by the liner layer, forming a replacement gate structure, removing the sacrificial gap fill material and forming a low-k sidewall spacer adjacent the liner layer. A device is also disclosed that includes a gate cap layer, a layer of silicon nitride or silicon oxynitride positioned on each of two upstanding portions of a gate insulation layer and a low-k sidewall spacer positioned on the layer of silicon nitride or silicon oxynitride.