This disclosure relates to systems and methods for tracking objects for augmented reality. Positions and/or orientations of objects, light sources, and/or visual markers may be determined. Visual effects may be determined based on the objects, lights sources, and/or visual markers. Overlay positions and/or overlay orientations for the visual effects may be determined based on the positions and orientations of the objects, light sources, and/or visual markers. An overlay image including the visual effects may be determined and displayed.