A well pump assembly has a number modules, including a pump and a motor, secured together. Each of the modules has a housing, and at least one of the modules has a neck of reduced diameter relative to the housing. A clamp is secured around the neck. The clamp has a dovetail groove extending around and within the outer surface of the clamp. An accessory member protrudes radially outward from the outer surface of the clamp and has an inner portion located in the groove. A threaded fastener secures the accessory member to the clamp. The accessory member is slidable along the groove to a selected orientation prior to tightening the fastener.