A handle assembly base has a central tube opening for receiving an elastic resistance tube with an expanded end diameter. A first block and a second block are pivotably mounted to the base on opposite sides of the central tube opening. When pivoted towards each other, the blocks define a reduced area opening through which the expanded end diameter of the resistance tube cannot pass, thereby retaining the resistance tube to the base. The base has two opposed ears which extend upwardly of the central tube opening. A flexible strap is secured to the ears, and extends through a cylindrical tube to which a flexible handle cushion is mounted to provide a grip for the user to engage the handle.