A well pump assembly has a tubular receptacle for attachment to a lower end of a string of production tubing. A docking station mounts to the receptacle, the docking station having a lower well fluid conduit for receiving well fluid from the well, a lower orientation guide, and a lower hydraulic connector. A hydraulic line extends alongside the receptacle and is in fluid communication with the lower hydraulic connector for supplying hydraulic fluid. A hydraulically actuated pump is configured to be lowered into and retrieved from the receptacle. A guide member on a lower end of the pump has an upper orientation guide, and an upper hydraulic connector in fluid communication with the pump, and an upper well fluid conduit leading to an intake of the pump.