The present disclosure provides various metrics that may be used to generate a score representing a conversational quality of a user/computing system dialog. User input data, corresponding to a dialog with a natural language processing system, may be determined and associated with a dialog identifier. System output data, corresponding to the dialog, may also be determined and associated with the dialog identifier. After the system output data is output to a user, the user input data and system output data may be processed to determine a score corresponding to the dialog exchange.