SYSTEM AND METHOD FOR AUTHORING AUGMENTED REALITY STORYTELLING EXPERIENCES INCORPORATING INTERACTIVE PHYSICAL COMPONENTS

    公开(公告)号:US20210312715A1

    公开(公告)日:2021-10-07

    申请号:US17223287

    申请日:2021-04-06

    Abstract: A unique storytelling system is disclosed herein in which a plug-and-play hardware platform with an integrated augmented reality (AR) environment brings stories to life. The storytelling system includes an electronics toolkit and a structure toolkit which enable the user to prototype interactive physical devices for storytelling. The interactive physical devices crafted by the user are easily programmed using a simple visual programing environment of the storytelling system. Additionally, a story event planning tool of the storytelling system enables the user to create customized interactions between the interactive physical devices and virtual AR objects, such as virtual AR avatars or the like. Finally, an AR storytelling application of the storytelling system utilizes an AR device, such as a smartphone, to bring the interactive physical devices to life and enable the user to tell stories using the custom interactions that he or she created.

    Method and System for Hand Pose Detection
    12.
    发明申请

    公开(公告)号:US20200225761A1

    公开(公告)日:2020-07-16

    申请号:US16707936

    申请日:2019-12-09

    Abstract: A method for hand pose identification in an automated system includes providing depth map data of a hand of a user to a first neural network trained to classify features corresponding to a joint angle of a wrist in the hand to generate a first plurality of activation features and performing a first search in a predetermined plurality of activation features stored in a database in the memory to identify a first plurality of hand pose parameters for the wrist associated with predetermined activation features in the database that are nearest neighbors to the first plurality of activation features. The method further includes generating a hand pose model corresponding to the hand of the user based on the first plurality of hand pose parameters and performing an operation in the automated system in response to input from the user based on the hand pose model.

    Method and System for Hand Pose Detection
    13.
    发明申请

    公开(公告)号:US20190310716A1

    公开(公告)日:2019-10-10

    申请号:US16436588

    申请日:2019-06-10

    Abstract: A method for hand pose identification in an automated system includes providing depth map data of a hand of a user to a first neural network trained to classify features corresponding to a joint angle of a wrist in the hand to generate a first plurality of activation features and performing a first search in a predetermined plurality of activation features stored in a database in the memory to identify a first plurality of hand pose parameters for the wrist associated with predetermined activation features in the database that are nearest neighbors to the first plurality of activation features. The method further includes generating a hand pose model corresponding to the hand of the user based on the first plurality of hand pose parameters and performing an operation in the automated system in response to input from the user based on the hand pose model.

    WINDOW-SHAPING VIRTUAL REALITY SYSTEM
    14.
    发明申请

    公开(公告)号:US20190035160A1

    公开(公告)日:2019-01-31

    申请号:US15925758

    申请日:2018-03-19

    Abstract: A virtual reality system, comprising an electronic 2d interface having a depth sensor, the depth sensor allowing a user to provide input to the system to instruct the system to create a virtual 3D object in a real-world environment. The virtual 3D object is created with reference to at least one external physical object in the real-world environment, with the external physical object concurrently displayed with the virtual 3D object by the interface. The virtual 3D object is based on physical artifacts of the external physical object.

    COLLABORATIVE 3D MODELING SYSTEM
    15.
    发明申请

    公开(公告)号:US20180122138A1

    公开(公告)日:2018-05-03

    申请号:US15821831

    申请日:2017-11-23

    Abstract: A collaborative 3D modeling system, comprising a computer processing unit, a digital memory, and an electronic display, the computer processing unit and the digital memory configured to provide 3D model representations of a first plurality of versions of an object component for a first user, the versions being selectable along a first axis, and using the electronic display, provide a plurality of user identifications which are selectable along a second axis, wherein selecting a subsequent user causes a second plurality of said versions of said object component to be displayed on the electronic display.

    MANIPULATING VIRTUAL ENVIRONMENT USING NON-INSTRUMENTED PHYSICAL OBJECT
    16.
    发明申请
    MANIPULATING VIRTUAL ENVIRONMENT USING NON-INSTRUMENTED PHYSICAL OBJECT 审中-公开
    使用非仪器物理对象操纵虚拟环境

    公开(公告)号:US20150331576A1

    公开(公告)日:2015-11-19

    申请号:US14712695

    申请日:2015-05-14

    CPC classification number: G06F3/04815 G06F3/011

    Abstract: A method of manipulating a three-dimensional image file including a virtual object includes obtaining image information in a processing device of a non-instrumented physical object manipulated by a user, such image information including movement information; and causing virtual movement of the virtual object based on the movement information. A method of shaping a virtual object includes obtaining image information including movement information; and determining a shape of the virtual object based on the movement information. A method of modifying a virtual object includes obtaining image information including movement information; and altering a virtual surface appearance of at least a part of the virtual object based on the movement information. Systems and computer-readable media are also described.

    Abstract translation: 一种操作包括虚拟对象的三维图像文件的方法包括:在由用户操纵的非仪器物理对象的处理设备中获取包括运动信息的图像信息的图像信息; 并基于移动信息引起虚拟对象的虚拟移动。 一种对虚拟对象进行整形的方法包括获取包括移动信息的图像信息; 以及基于所述移动信息来确定所述虚拟对象的形状。 修改虚拟对象的方法包括获取包括移动信息的图像信息; 以及基于所述移动信息来改变所述虚拟对象的至少一部分的虚拟表面外观。 还描述了系统和计算机可读介质。

    Authoring systems and methods for enabling bidirectional binding of augmented reality with toys in real-time

    公开(公告)号:US12288299B2

    公开(公告)日:2025-04-29

    申请号:US18480158

    申请日:2023-10-03

    Abstract: An augmented reality (AR) interaction authoring system is described. The AR interaction authoring system is configured to support the real-time creation of AR applications to support AR-enhanced toys. The design of the AR interaction authoring system enables bidirectional interactions between the physical-virtual space of toys and AR. The AR interaction authoring system allows intuitive authoring of AR animations and toys actuations through programming by demonstration, while referring to the physical toy for a contextual reference. Using a visual programming interface, users can create bidirectional interactions by utilizing users' input on toys to trigger AR animations and vice versa. A plug-and-play IoT toolkit is also disclosed that includes hardware to actuate common toys. In this way, users can effortlessly integrate toys into the virtual world in an impromptu design process, without lengthy electronic prototyping.

    UBIQUITOUS TANGIBLE OBJECT UTILIZATION THROUGH CONSISTENT HAND-OBJECT INTERACTION IN AUGMENTED REALITY

    公开(公告)号:US20250124671A1

    公开(公告)日:2025-04-17

    申请号:US18908885

    申请日:2024-10-08

    Abstract: The disclosed system and method enable hand-object interaction with a virtual object in augmented reality or virtual reality. The system and method advantageously suggest particular physical objects in the environment to be used as physical proxies for virtual objects to be interacted with. The system and method maintain physical and mental consistency in the user experience by recommending physical proxies in a manner that takes into consideration the interaction constraints. Finally, the system and method advantageously incorporate a mapping process that takes into consideration the object, the hand gesture, and the contact points on both the physical and virtual object, thereby providing consistent visualization of the virtual hand-object interactions to the users.

    POSE RELATION TRANSFORMER AND REFINING OCCLUSIONS FOR HUMAN POSE ESTIMATION

    公开(公告)号:US20240296582A1

    公开(公告)日:2024-09-05

    申请号:US18584191

    申请日:2024-02-22

    Abstract: An approach for pose estimation is disclosed that can mitigate the effect of occlusions. A POse Relation Transformer (PORT) module is configured to reconstruct occluded joints given the visible joints utilizing joint correlations by capturing the implicit joint occlusions. The PORT module captures the global context of the pose using self-attention and a local context by aggregating adjacent joint features. To train the PORT module to learn joint correlations, joints are randomly masked and the PORT module learns to reconstruct the masked joints, referred to as Masked Joint Modeling (MJM). Notably, the PORT module is a model-agnostic plug-in for pose refinement under occlusion that can be plugged into any existing or future keypoint detector with substantially low computational costs.

Patent Agency Ranking