Technologies are described for providing a projected augmented reality system with pose tracking for directing manual processes. A projected augmented reality system includes a video projector, configured to project a dynamically-changing image onto a surface within a work area of an operator, a sensor, and a computer. The computer includes a memory including instructions that when executed cause the computer to obtain three-dimensional pose data using the sensor, determine an output graphical element based on a sequence of three-dimensional pose data over time, and on a current production state, and send an output image based on the output graphical element to the video projector for projection onto the surface.