Preventing the loss of physical objects using cognitive computing includes receiving, by a wearable device, information associated with a user of the wearable device. The information received includes one or more gestures to be used as a digital handshake and at least one physical object to be tracked. A set of models associated with the at least one physical object is downloaded to the wearable device. In response to an activation command, the wearable device performs the digital handshake with the at least one object to be tracked. The wearable device continuously monitors actions and movements performed by the user to determine whether the at least one physical object is in possession of the user.