A syringe holder includes a body with a syringe retainer, an elongated finger and a stop member connected to the body. The finger inserts longitudinally into a socket on a pump without the aid of tools and permits the holder to move in multiple planes. The stop member limits such movement. The stop member can include a clip for releasably clamping onto a portion of a pump. The syringe retainer can include a hole in the body and may optionally include an adaptor mounted in or extending into the hole for detachably attaching a syringe. The syringe holder can be incorporated into a tube set assembly or an infusion system.