TOKEN PRUNING IN SWIN TRANSFORMER ARCHITECTURES

    公开(公告)号:US20240221375A1

    公开(公告)日:2024-07-04

    申请号:US18400635

    申请日:2023-12-29

    CPC classification number: G06V10/94 G06V10/778

    Abstract: Token pruning in Swin transformer architectures is provided via identifying initial windows into which the tokenized input to a Swin transformer architecture is divided and a pruning target; identifying D1 tokens in each initial window, excluding those tokens located in a first row of each initial window, having a lowest information content; merging each of the D1 tokens in each initial window into another token in that initial window in a vertical direction to transform each initial window into a corresponding intermediate window; identifying D2 tokens in each intermediate window, excluding those tokens located in a first column of each intermediate window, having a lowest information content; merging each of the D2 tokens in each intermediate window into another token in that intermediate window in a horizontal direction to transform each intermediate window into a corresponding spatially complete window.

Patent Agency Ranking