An efficient technique to build subject-specific skeleton models from external measurements is provided. A generic human skeleton model is manipulated and deformed using marker positional data from human motion-captured sequences. The joint and bone geometry parameters are optimized to construct an estimate of the specific skeleton model of the human subject that originally performed the motion sequence. The scope of the fitting procedure can be adjusted to coincide with a series of smaller optimizations over local bone regions or expanded to a single global optimization over all skeleton geometry and joint parameters simultaneously.