Image labeling with global parameters is described. In an embodiment a pose estimation system executes automatic body part labeling. For example, the system may compute joint recognition or body part segmentation for a gaming application. In another example, the system may compute organ labels for a medical imaging application. In an example, at least one global parameter, for example body height is computed for each of the images to be labeled. In an example, the global parameter is used to modify an image labeling process. For example the global parameter may be used to modify the input image to a canonical scale. In another example, the global parameter may be used to adaptively modify previously stored parameters of the image labeling process. In an example, the previously stored parameters may be computed from a reduced set of training data.