A call assistant device is used to command a call management system to perform a specified task in association with a specified call. The call assistant device can be an Internet of Things (IoT) based device, which can include one or more buttons and connect to a communication network wirelessly. When a user activates the call assistant device, e.g., presses a button, the call assistant device sends a message to the call management system to perform a specified task. Upon receiving the message, the call management system executes the specified task in association with a specified call of the user. The task to be performed can be any task that can be performed in association with a call, e.g., generating a summary of the call, bookmarking a specified moment in the call, sending a panic alert to a particular user, or generating an action item.