A system and method to perform incremental adaptive modification of a virtual reality image involve obtaining sensor measurements from sensors coupled to an individual who is performing a task while viewing the virtual reality image. The method includes generating a true model of the individual from the sensor measurements, comparing the true model with an expert model obtained from another individual performing the task, and developing a lesson plan based on determining a difference between the true model and the expert model. The developing the lesson plan includes determining a gradient of intermediate models within the difference. A different one of the intermediate models is included iteratively along the gradient from the true model to the expert model in the virtual reality image.