An anchoring device for anchoring a rod in a bone or a vertebra and for use with at least two rods having a different diameter is provided. The anchoring device includes a shaft and a head for connection to one of said rods. The head is connected to said shaft and has a recess with a base and two substantially vertically extending legs defining a channel for receiving the rod in said channel. The anchoring device further includes a fixation element for clamping said rod in said channel. The fixation element and said base each have a contact surface contacting said rod, the contact surfaces being shaped so as to be able to clamp any of the different rods.