Described is a system for detecting adversarial activities. During operation, the system generates a multi-layer temporal graph tensor (MTGT) representation based on an input tag stream of activities. The MTGT representation is decomposed to identify normal activities and abnormal activities, with the abnormal activities being designated as adversarial activities. A device can then be controlled based on the designation of the adversarial activities.