Systems and methods for tracking activities from a plurality of multimodal inputs are described. Activity tracking can include receiving a plurality of multimodal inputs, synchronizing the plurality of multimodal inputs, generating segments from the synchronized multimodal inputs, recognizing activities associated with each generated segment by performing a bagged formal concept analysis (BFCA), and recording the recognized activities in a storage. Tracking of activities can include the detection of moments (e.g., eating moments), during which an activity tracking application can prompt a user for information (e.g., a food journal).