A gripper assembly for anchoring a tool within a downhole passage and for possibly assisting movement of the tool within the passage. The gripper assembly includes an elongated mandrel and flexible toes that can be radially displaced to grip onto the surface of the passage. The toes are displaced by the interaction of a driver slidable on the mandrel and a driver interaction element on the toes. In one embodiment, the toes are displaced by the interaction of rollers and ramps that are longitudinally movable with respect to one another. In another embodiment, the toes are displaced by the interaction of toggles that rotate with respect to the toes.