Example implementations described herein are directed to systems and methods for non-invasive data extraction from digital displays. In an example implementation, a method includes receiving one or more video frames from a video capture device capturing an external display, where the external display is independent the video capture device; determining one or more locations within the external display comprising time varying data of the external display; and for each identified location of the time varying data: determining a data type; applying one or more rules based on the data type; and determining an accuracy of the time varying data within the one or more frames based on the rules.