An armrest attaching structure for attaching an armrest to a vehicle seat having a seat back frame, the seat back frame having a side frame element that is connected to a base member at a lower end portion thereof, may include an armrest attachment member having an armrest attaching portion and a connecting portion. The armrest attaching portion is connected to the armrest. The connecting portion is connected to the side frame element of the seat back frame at a position closer to the lower end portion of the side frame element than the armrest attaching portion.