An in-call virtual assistant system monitors a real-time call, e.g., a call that is in progress, between multiple speakers, identifies a trigger and executes a specified task in response to the trigger. The virtual assistant system can be invoked by an explicit trigger or an implicit trigger. For example, an explicit trigger can be a voice command from one of the speakers in the call, such as “Ok Chorus, summarize the call” for summarizing the call. An implicit trigger can be an event that occurred in the call, or outside of the call but that is relevant to a speaker. For example, an event such as a speaker dropping off the call suddenly can be an implicit trigger that invokes the virtual assistant system to perform an associated task, such as notifying the remaining speakers on the call that one of the speakers dropped.