Invention Grant
- Patent Title: Text-conditioned image search based on transformation, aggregation, and composition of visio-linguistic features
-
Application No.: US17160893Application Date: 2021-01-28
-
Publication No.: US11720651B2Publication Date: 2023-08-08
- Inventor: Pinkesh Badjatiya , Surgan Jandial , Pranit Chawla , Mausoom Sarkar , Ayush Chopra
- Applicant: Adobe Inc.
- Applicant Address: US CA San Jose
- Assignee: Adobe Inc.
- Current Assignee: Adobe Inc.
- Current Assignee Address: US CA San Jose
- Agency: Finch & Maloney PLLC
- Main IPC: G06F18/25
- IPC: G06F18/25 ; G06N3/04 ; G06F16/583 ; G06F16/532 ; G06F16/538 ; G06F18/214

Abstract:
Techniques are disclosed for text-conditioned image searching. A methodology implementing the techniques includes decomposing a source image into visual feature vectors associated with different levels of granularity. The method also includes decomposing a text query (defining a target image attribute) into feature vectors associated with different levels of granularity including a global text feature vector. The method further includes generating image-text embeddings based on the visual feature vectors and the text feature vectors to encode information from visual and textual features. The method further includes composing a visio-linguistic representation based on a hierarchical aggregation of the image-text embeddings to encode visual and textual information at multiple levels of granularity. The method further includes identifying a target image that includes the visio-linguistic representation and the global text feature vector, so that the target image relates to the target image attribute, and providing the target image as an image search result.
Public/Granted literature
Information query