Systems and methods are provided for three dimensional depth reconstruction of vessels in two dimensional medical images. A medical image comprising braches of one or more vessels is received. A branch overlap image channel that represents a pixelwise probability that the branches overlap is generated. A set of branch orientation image channels are generated. Each branch orientation image channel is associated with one of a plurality of orientations. Each branch orientation image channel representing a image channel represents a pixelwise probability that the branches are oriented in its associated orientation. A multi-channel depth image is generated based on the branch overlap image channel and the set of branch orientation image channels. Each channel of the multi-channel depth image comprises portions of the branches corresponding to a respective depth.