The present disclosure provides a retaining wall fall protection system having a plurality of base plates. It further includes a plurality of uprights pivotally attached to the base plates and a cross-brace attached to adjacent uprights at opposite ends of the cross-brace. A plurality of guardrails are provided and attached to guardrail adjustment brackets. The guardrail brackets are removably attachable to the uprights at various positions along the length of the uprights. Finally, there is provided an attachment assembly which secures the uprights in position adjacent the retaining wall. The attachment assembly comprises an attachment strap which is mounted to the upright and sized to extend through the retaining wall. A retainer on the opposite side of the wall engages a distal end of the attachment strap to secure the scaffolding to the retaining wall under construction.