A system (100) and method is provided for performing a model-based segmentation of an anatomical structure in a medical image of a patient. The medical image (022) is accessed. Moreover, model data (162) is provided which defines a deformable model for segmenting the type of anatomical structure. The model-based segmentation of the anatomical structure is performed by adapting the deformable model to the anatomical structure in the medical image using an adaptation technique. In accordance with the present invention, performing the model based segmentation further comprises determining from patient data (042) medical information which is predictive of an appearance of the anatomical structure in the medical image, and adjusting or setting a segmentation parameter based on the medical information so as to adjust the model-based segmentation to said predicted appearance of the anatomical structure in the medical image, the segmentation parameter being a parameter of i) the deformable model or ii) the adaptation technique. Advantageously, the system and method are enabled to better cope with the inter-patient and inter-disease-stage variability in the appearance of anatomical structures.