Systems and methods for automated assessment of a vessel are provided. One or more input medical images of a vessel of a patient are received. A plurality of vessel assessment tasks for assessing the vessel is performed using a machine learning based model trained using multi-task learning. The plurality of vessel assessment tasks comprises segmentation of reference walls of the vessel from the one or more input medical images and segmentation of lumen of the vessel from the one or more input medical images. Results of the plurality of vessel assessment tasks are output.