A target recognition system and method. The inventive method includes the steps of first receiving an image of a scene within which a target may be included and second using constrained image segmentation as a basis for target recognition. In the specific embodiment, the segmentation is variable and derived from target geometry models. The constrained image segmentation is adapted to use target geometry models to derive alternative image segmentation hypotheses. The hypotheses are verified using a novel hierarchical adaptive region model matcher. In the best mode, the hierarchical adaptive region model matcher is fully hierarchical and includes the steps of receiving an image and a hypothesis with respect to a target in the image. In the illustrative embodiment, the hypothesis contains the type, location, and/or orientation of a hypothesized target in the image. The hypothesis is received by a model tree generator. A target model library provides a plurality of computer generated target models. The model tree generator retrieves a model from the library based on a hypothesized target type and renders a model tree therefrom that represents the appearance of a hypothesized target at a hypothesized location and orientation. In the illustrative embodiment, the verification model further includes a search manager adapted to receive a model tree from the model tree generator. A feature extractor is included along with a segmentation evaluator. The segmentation evaluator is coupled to receive image features from the feature extractor. The segmentation evaluator is adapted to output a segmentation score to the search manager with respect to a specified set of segmentation labels based on a supplied set of image features for an active node.