The present disclosure relates to a method and apparatus for training a convolutional neural network (CNN) that is robust to missing input information. The method includes: receiving multiple three-dimensional (3D) images per case obtained by different imaging systems such as computed tomography (CT), magnetic resonance (MR), and positron emission tomography (PET); processing the 3D images to fuse the information from multiple imaging modalities; building a deep learning framework using CNNs for image segmentation; adapting the deep learning framework to handle either a single missing input modality or multiple modalities by emulating missing modalities in training; post-processing the output from the deep learning framework to obtain the final segmentation.