The present teaching generally relates to detecting data bucket discrepancies associated with online experiments. In a non-limiting embodiment, a monitoring layer may be generated within an online experimentation platform that includes at least a first layer, and where a first online experiment is associated with the first layer, the monitoring layer includes a monitoring layer data bucket, and the first layer includes at least a first data bucket. First data representing user activity associated with a first plurality of identifiers may be obtained, the user activity being associated with the first layer. Second data representing at least one user engagement parameter may be generated, and a first discrepancy between the first and second data may be determined. The first discrepancy indicating a first amount of identifiers that include a first metadata tag associated with the first layer and lack a second metadata tag associated with the monitoring layer.