A scalable tracking system includes a camera subsystem, a weight subsystem, and a central server. The camera subsystem includes cameras that capture video of a space, camera clients that determine local coordinates of people in the captured videos, and a camera server that determines the physical positions of people in the space based on the determined local coordinates. The weight subsystem determines when items were removed from shelves. The central server determines which person in the space removed the items based on the physical positions of the people in the space and the determination of when items were removed.