Systems and methods for determining a 3D centerline of a vessel are provided. A current state observation of an artificial agent is determined based on one or more image view sets, each including 2D medical images of a vessel, a current position of the artificial agent in the 2D medical images, and a start position and a target position in the 2D medical images. Policy values are calculated for a plurality of actions for moving the artificial agent in 3D based on the current state observation using a trained machine learning model. The artificial agent is moved according to a particular action based on the policy values. The steps of determining, calculating, and moving are repeated for a plurality of iterations to move the artificial agent along a 3D path between the start position and the target position. The 3D centerline of the vessel is determined as the 3D path.