A packer features spaced apart sealing elements with an extrusion barrier between them. When the packer is set the extrusion barrier is protected from debris in the well. The barrier provides full circumferential extrusion protection using one or more rings made of wedge shaped segments that have a keyway at their edges and are assembled in an alternating manner so as to be able to increase or decrease in diameter when mandrel components are moved toward or away from each other. The segments have an opening through which a mandrel projection extends so as to force the segments into the smaller diameter for removal. Travel stops for the segments in the form of machined flats are provided on the relatively movable mandrel components.