Semi-tubular self-piercing rivets and solid rivets are provided with surface features such as recesses or lobes. The recesses or lobes engage the panels that are joined by the rivet to prevent rotation of one or both panels after rivet insertion. The recesses or lobes may be provided at the head end or tip end of the rivet or may be provided at both the head end and the tip end of the rivet.