A system and method are disclosed for segmenting a set of two-dimensional CT slices corresponding to a lesion. In an embodiment, for each of at least a subset of the set of CT slices, the system inputs the CT slice into a plurality of branches of a trained segmentation block. Each branch of the segmentation block includes a convolutional neural network (CNN) with filters at a different scale, and produces a plurality of levels of output. The system generates, for each CT slice in the subset, feature maps for each level of output. The system generates a segmentation of each CT slice in the subset based on the feature maps of each level of output. The system aggregates the segmentations of each slice in the subset to generate a three-dimensional segmentation of the lesion. The system provides data representing the three-dimensional segmentation for display.