According to aspects illustrated herein, there is provided a method, system, and computing device for identifying bugs in a software application. The method provides for hosting the software application on a cloud server and providing the software application to a group of users and monitors usage of the software application. Next, new usage patterns are identified by identifying current usage patterns based on the monitoring are identified and comparing the current usage patterns to stored usage patterns. After that, bugs are identified by: 1) automatically generating a set of new test matrices based on the new usage patterns; 2) automatically running the set of new test matrices on the software application, while the software application is being used; 3) recording error messages generated; and 4) correlating the error messages with the new usage patterns to identify the bugs. Then, a reporting message is generated to report the identified bugs.