One embodiment of the invention sets forth a technique for shading a graphics object inserted into a video feed of a real-world scene based on lighting conditions in the real-world scene. The real-world scene includes a fiducial marker denotes the location in the video feed where the graphics object should be inserted. In order to shade the graphics object, the AR application computes light color values at multiple points on the fiducial marker. The color computation module also computes the direction of light cast on the fiducial marker by determining the direction of the shadow cast by a pyramid object on the fiducial marker. The AR application then shades the graphics object inserted into the video feed at the location of the fiducial marker based on the light color values and the direction of light.