A system and method for motion capture to efficiently render in-game objects, including characters, and controlling animation of characters and non-player characters. In some embodiments, the motion capture includes a machine learning model that can take as inputs information about the hand pose of a person from multiple sensors over a period of time and generates a highly probable hand pose prediction from that information, based on what it learns from a previously captured dataset of high quality sample motion captures. The systems and methods described herein overcome the imprecision conventional optical marker pipelines for the case of finger movement, where the slightest imprecision yields results that are not good enough for production.