In general, in one aspect, the invention relates to a method for managing data, the method includes obtaining, by a data management system, a first resource usage of a local data system associated with a first data confidence fabric (DCF) pipeline, wherein the first DCF pipeline is associated with a first workload, obtaining second resource usage of the local data system associated with a second DCF pipeline, wherein the second DCF pipeline is associated with a second workload, analyzing the first resource usage and the second resource usage to obtain a workload ranking, and performing an action set based on the workload ranking.