A headrest support structure of a conveyance seat includes: an upper frame that is provided with a pair of fitting holes and that constitutes a back frame; and a pair of cylindrical headrest holders, each of which fits in one of the pair of fitting holes to be fixed to the upper frame. A transverse cross-section of a fitting portion of each headrest holder fitting in the corresponding fitting hole has an asymmetric shape about a fitting axis parallel to a fitting direction in which the headrest holder fits in the fitting hole, the transverse cross-section being a cross-section in a direction orthogonal to the fitting axis. The pair of fitting holes is formed such that each headrest holder is allowed to fit in only one of the pair of fitting holes provided at a position set as a fixing position of the headrest holder.