A pipe tally vision system may be positioned on a drilling rig. A drill string may be positioned within a wellbore using the drilling rig. The drill string may be observed with the pipe tally system during a wellbore operation. A pipe tally database may be generated by the pipe tally system based on the observations of the drill string made by the pipe tally system and observations may be made as the drill string is moved into or out of the wellbore.