公开(公告)号：US20250131027A1

公开(公告)日：2025-04-24

申请号：US18924763

申请日：2024-10-23

Applicant: SRI International

Inventor： Yangyi Chen ,  Karan Sikka ,  Michael A. Cogswell ,  Ajay Divakaran

IPC: G06F16/338 ,  G06F16/33 ,  G06F16/532

Abstract: In an example, a method for fine-tuning a Large Visual Language Model (LVLM) includes providing visual queries, each of the visual queries comprises at least an image and a textual query related to the image; processing, by the LVLM, the visual queries to extract visual embeddings from the visual queries, wherein the LVLM comprises a Visual Language Model (VLM), a first Large Language Model (LLM), and a linear projection layer interconnecting the VLM and the LLM; for visual queries: i) generating, by the LVLM, a response to the corresponding visual query based on the corresponding visual embedding; ii) evaluating, by a second LLM, the generated response to verify that the generated response satisfies predefined criteria; and iii) providing, by the second LLM, a feedback to the LVLM, in response to the evaluating the generated response; and fine-tuning the LVLM using aggregated feedback provided by the second LLM for the visual queries.

公开(公告)号：US20240403649A1

公开(公告)日：2024-12-05

申请号：US18520800

申请日：2023-11-28

Applicant: SRI International

Inventor： Han-Pang Chiu ,  Yi Yao ,  Zachary Seymour ,  Alex Krasner ,  Bradley J. Clymer ,  Michael A. Cogswell ,  Cecile Eliane Jeannine Mackay ,  Alex C. Tozzo ,  Tixiao Shan ,  Philip Miller ,  Chuanyong Gan ,  Glenn A. Murray ,  Richard Louis Ferranti ,  Uma Rajendran ,  Supun Samarasekera ,  Rakesh Kumar ,  James Smith

IPC: G06N3/0895

Abstract: In an example, a system includes processing circuitry in communication with storage media. The processing circuitry is configured to execute a machine learning system including at least a first module, a second module and a third module. The machine learning system is configured to train one or more machine learning models. The first module is configured to generate augmented input data based on the streaming input data. The second module includes a machine learning model configured to perform a specific task based at least in part on the augmented input data. The third module configured to adapt a network architecture of the one or more machine learning models based on changes in the streaming input data.

公开(公告)号：US11934793B2

公开(公告)日：2024-03-19

申请号：US17516409

申请日：2021-11-01

Applicant: SRI International

Inventor： Ajay Divakaran ,  Karan Sikka ,  Yi Yao ,  Yunye Gong ,  Stephanie Nunn ,  Pritish Sahu ,  Michael A. Cogswell ,  Jesse Hostetler ,  Sara Rutherford-Quach

IPC: G06F40/35 ,  G06F16/33 ,  G06N5/04

CPC classification number: G06F40/35 ,  G06F16/3335 ,  G06N5/04

Abstract: A method, apparatus and system for training an embedding space for content comprehension and response includes, for each layer of a hierarchical taxonomy having at least two layers including respective words resulting in layers of varying complexity, determining a set of words associated with a layer of the hierarchical taxonomy, determining a question answer pair based on a question generated using at least one word of the set of words and at least one content domain, determining a vector representation for the generated question and for content related to the at least one content domain of the question answer pair, and embedding the question vector representation and the content vector representations into a common embedding space where vector representations that are related, are closer in the embedding space than unrelated embedded vector representations. Requests for content can then be fulfilled using the trained, common embedding space.

公开(公告)号：US20250131212A1

公开(公告)日：2025-04-24

申请号：US18919630

申请日：2024-10-18

Applicant: SRI International

Inventor： Pengfei Yu ,  Yi Yao ,  Karan Sikka ,  Michael A. Cogswell ,  Ajay Divakaran

IPC: G06F40/56

Abstract: In an example, a method for generating responses by a Machine Learning (ML) system includes processing, by a first language model, a natural language instruction to generate an instruction representation based on a meaning of the natural language instruction; translating, by a translation module comprising an interface between the first language model and a second language model, the instruction representation into data indicating an intent of the natural language instruction, wherein the second language model is trained with domain specific knowledge; providing, by the translation module, the natural language instruction and the data indicating the intent of the natural language instruction to the second language model; and generating, by the second language model, a response based on the natural language instruction and the data indicating the intent of the natural language instruction.

公开(公告)号：US20240005654A1

公开(公告)日：2024-01-04

申请号：US17656391

申请日：2022-03-24

Applicant: SRI International

Inventor： Arijit Ray ,  Michael A. Cogswell ,  Ajay Divakaran ,  Yi Yao ,  Giedrius T. Burachas ,  Kamran Alipour

IPC: G06V10/98 ,  G06T11/00 ,  G06V10/776 ,  G06V10/77

CPC classification number: G06V10/98 ,  G06T11/001 ,  G06V10/776 ,  G06V10/7715

Abstract: A computing system comprising a memory configured to store an artificial intelligence (AI) model and an image, and a computation engine executing one or more processors may be configured to perform the techniques for error-based explanations for AI behavior. The computation engine may execute the AI model to analyze the image to output a result. The AI model may, when analyzing the image to output the result, process, based on data indicative of the result, the image to assign an error score to each image feature extracted from the image, and obtain, based on the error scores, an error map. The AI model may next update, based on the error map and to obtain a first updated image, the image to visually indicate the error score assigned to each of the image features, and output one or more of the error scores, the error map, and the first updated image.