A golf club head, preferably a putter head, comprising a replaceable face insert is disclosed herein. The golf club head comprises a body with face cavity and a secondary cavity in communication with, and disposed behind, the face cavity, and at least one opening extending through a top or sole portion into the secondary cavity. The replaceable face insert comprises a striking portion and a support structure having an elongated, vertical through bore extending from a rear surface of the striking portion. The support structure is received within the secondary cavity and the striking portion is received within the face cavity, and then a rod or bolt is inserted through the opening and into the elongated, vertical through bore to reversibly secure the face insert to the body. A linear bearing may be disposed within the elongated through bore to receive the rod or bolt.