An ultrahigh pressure pump includes a frame; a crankshaft having a journal; and at least one telescoping pump subassembly having inner and outer ends. The outer end is carried by the frame pivot so as to allow pivotal swinging movement of the pump subassembly, and the inner end is attached to the journal. The piston rod can reciprocate relative to the inner bore substantially free from side loads. The pump subassembly includes: an outer member including a cylinder defining an inner bore; and a inner member having a piston rod and a outer sleeve. The piston rod is received in the inner bore and the cylinder is received in the outer sleeve. First and second restraining elements are disposed at spaced-apart positions along the axis of the pump subassembly and are configured to oppose misalignment forces between the piston rod and the cylinder.