A polyaxial bone anchor with a receiver, a shank, a pressure insert, a retainer for holding the shank in the receiver, and a closure. The receiver having at least one vertically aligned guide on a surface of an interior shank and insert receiving chamber. The insert having at least one shoulder that aligns with the guide and both properly positions the insert relative to the receiver during assembly and prevents the insert from axially rotating relative to the receiver.