An AI-based process monitoring system access a plurality of data sources having different data formats to collect and analyze KPI data and shortlist KPIs that are to be used for determining the impact of automation of an automated process or sub-process. Information regarding an automated process is received and KPIs associated with the process and sub-processes of the process are identified. The identified KPIs are put through an approval process and the approved KPIs are presented to a user for selection. The user-selected KPIs are evaluated based on classification, ranking and sentiments associated therewith. The evaluations are again presented to the user along with a set of questionnaires wherein each of the questions has a dynamically controlled weight associated therewith. Based at least on the weights and user responses, a subset of the evaluated KPIs are shortlisted for use in evaluating the impact of process automation.