Counterfactual Policy Evaluation of Model Performance

    公开(公告)号:US20240220859A1

    公开(公告)日:2024-07-04

    申请号:US18393349

    申请日:2023-12-21

    Applicant: Maplebear Inc.

    CPC classification number: G06N20/00 G06Q30/01

    Abstract: An online system uses an offline iterative clustering process to evaluate the performance of a set of content selection frameworks. To perform an iteration of the iterative clustering process, an online system clusters the testing example data into a set of clusters. An online system computes a set of framework scores for each of the generated clusters. An online system computes an improvement score for each cluster based on the performance scores of the clusters. To determine whether to perform another iteration, an online system computes an aggregated improvement score based on the improvement scores of the clusters. If an online system determines that the aggregated improvement score does not meet the threshold, an online system performs another iteration of the process above. When an online system finishes the iterative process, an online system outputs the improvement scores of the most-recent iteration.

Patent Agency Ranking