Systems and methods are described for improving connection to and management of conference calls. An inference/rules engine and human backed speech interpretation subsystem enable the system to efficiently use knowledge about the call to identify the caller and determine to which conference he or she should be connected. During the conference, a virtual assistant is available to interpret and execute system commands spoken by a caller, such as in a side conference accessed via DTMF tones.