A swelling packer is covered with a material that is preferably in a tubular form and slipped over the swelling element to be shrink fit with applied heat. The material is formulated to break down at temperatures slightly below the expected downhole temperatures so that ideally the packer has about 48 hours of swelling delay which is normally a time period long enough to allow it to be properly located without it swelling so much as to cause it to be damaged by running in. Various polymers can be used to make an imperious covering for run in that on the way starts to break down. Ideally the material for the cover disappears about the time of sealing or shortly thereafter.