A system including an image sensor that captures image data of a gaming table and a player area, and a tracking controller communicatively coupled to the image sensor. The tracking controller detects a player and a token set from the captured image data by applying an image neural network model to the image data to generate at least one key player data element for the player and at least one key token data element for the token set, generates a player data object representing physical characteristics of the player based on the key player data elements, links the player data object to a player identifier of the player, generates a token identifier based on the key token data elements, and links the token identifier to the player data object based on a physical relationship between the player and the token set indicated by the key data elements.