Surface UI for gesture-based interaction

    公开(公告)号:US20060036944A1

    公开(公告)日:2006-02-16

    申请号:US10914922

    申请日:2004-08-10

    Applicant: Andrew Wilson

    Inventor: Andrew Wilson

    CPC classification number: G06F3/0425 G06F3/04883

    Abstract: Disclosed is a unique system and method that facilitates gesture-based interaction with a user interface. The system involves an object sensing configured to include a sensing plane vertically or horizontally located between at least two imaging components on one side and a user on the other. The imaging components can acquire input images taken of a view of and through the sensing plane. The images can include objects which are on the sensing plane and/or in the background scene as well as the user as he interacts with the sensing plane. By processing the input images, one output image can be returned which shows the user objects that are in contact with the plane. Thus, objects located at a particular depth can be readily determined. Any other objects located beyond can be “removed” and not seen in the output image.

    System and process for controlling electronic components in a ubiquitous computing environment using multimodal integration

    公开(公告)号:US06990639B2

    公开(公告)日:2006-01-24

    申请号:US10160659

    申请日:2002-05-31

    Applicant: Andrew Wilson

    Inventor: Andrew Wilson

    Abstract: The present invention is directed toward a system and process that controls a group of networked electronic components using a multimodal integration scheme in which inputs from a speech recognition subsystem, gesture recognition subsystem employing a wireless pointing device and pointing analysis subsystem also employing the pointing device, are combined to determine what component a user wants to control and what control action is desired. In this multimodal integration scheme, the desired action concerning an electronic component is decomposed into a command and a referent pair. The referent can be identified using the pointing device to identify the component by pointing at the component or an object associated with it, by using speech recognition, or both. The command may be specified by pressing a button on the pointing device, by a gesture performed with the pointing device, by a speech recognition event, or by any combination of these inputs.

    Using size and shape of a physical object to manipulate output in an interactive display application

    公开(公告)号:US20060007123A1

    公开(公告)日:2006-01-12

    申请号:US10879872

    申请日:2004-06-28

    CPC classification number: G06F3/0425

    Abstract: Output of a computer system is manipulated using a physical object disposed adjacent to an interactive display surface. A painting application produces an image in response to an object disposed adjacent to the interactive display surface. During each of a plurality of capture intervals, a set of points corresponding to the object is detected when the object is disposed adjacent to the interactive display surface. An image is projected onto the interactive display surface representing the set of points and is filled with a color or pattern. As successive sets of points are accumulated during each of a plurality of capture intervals, a composite image is displayed. An object can thus be used, for example, to “draw,” “paint,” or “stamp” images on the display surface. These images manifest characteristics of the object and its interaction and movement relative to the interactive display surface in a realistic manner.

    Interaction between objects and a virtual environment display
    75.
    发明申请
    Interaction between objects and a virtual environment display 有权
    对象与虚拟环境之间的交互显示

    公开(公告)号:US20050245302A1

    公开(公告)日:2005-11-03

    申请号:US10834675

    申请日:2004-04-29

    Abstract: An interactive table has a display surface on which a physical object is disposed. A camera within the interactive table responds to infrared (IR) light reflected from the physical object enabling a location of the physical object on the display surface to be determined, so that the physical object appear part of a virtual environment displayed thereon. The physical object can be passive or active. An active object performs an active function, e.g., it can be self-propelled to move about on the display surface, or emit light or sound, or vibrate. The active object can be controlled by a user or the processor. The interactive table can project an image through a physical object on the display surface so the image appears part of the object. A virtual entity is preferably displayed at a position (and a size) to avoid visually interference with any physical object on the display surface.

    Abstract translation: 交互式表具有设置物理对象的显示面。 交互式表中的相机响应于从物理对象反射的红外(IR)光,从而能够确定显示表面上的物理对象的位置,使得物理对象显示在其上显示的虚拟环境的一部分。 物理对象可以是被动的或主动的。 活动对象执行主动功能,例如,其可以自推进在显示表面上移动,或发出光或声音或振动。 活动对象可由用户或处理器控制。 交互式表格可以通过显示表面上的物理对象投影图像,从而将图像显示在对象的一部分中。 虚拟实体优选地在位置(和尺寸)处显示,以避免与显示表面上的任何物理对象的视觉干扰。

    Template matching on interactive surface
    76.
    发明申请
    Template matching on interactive surface 审中-公开
    交互式表面上的模板匹配

    公开(公告)号:US20050227217A1

    公开(公告)日:2005-10-13

    申请号:US10813855

    申请日:2004-03-31

    Applicant: Andrew Wilson

    Inventor: Andrew Wilson

    CPC classification number: G06K9/20 G06K9/6203

    Abstract: A patterned object that is placed on or adjacent to a display surface of an interactive display is detected by matching an image produced using infrared light reflected from the patterned object with one of a set of templates associated with the patterned object. The templates are created for each of a plurality of incremental rotations of the patterned object on a display surface. To implement the comparison, a sum of template data value corresponding to the intensities of the reflected light is calculated for the image of the patterned object and for each of the templates. These sums are compared to determine a rotated template that matches the patterned object within a predefined threshold, thus determining that the patterned object has been placed on or near the display surface.

    Abstract translation: 通过将从图案化对象反射的红外光产生的图像与与图案化对象相关联的一组模板中的一个匹配,来检测放置在交互式显示器的显示表面上或附近的图案化对象。 为显示表面上的图案化对象的多个增量旋转中的每一个创建模板。 为了实现比较,针对图案化对象的图像和每个模板计算与反射光的强度相对应的模板数据值的总和。 将这些和进行比较以确定与预定阈值内的图案化对象匹配的旋转模板,从而确定图案化对象已被放置在显示表面上或附近。

    Determining connectedness and offset of 3D objects relative to an interactive surface
    77.
    发明申请
    Determining connectedness and offset of 3D objects relative to an interactive surface 有权
    确定3D对象相对于交互式表面的连接性和偏移量

    公开(公告)号:US20050226505A1

    公开(公告)日:2005-10-13

    申请号:US10814761

    申请日:2004-03-31

    Applicant: Andrew Wilson

    Inventor: Andrew Wilson

    CPC classification number: G06K9/00375 G06F3/0425 G06F2203/04101

    Abstract: A position of a three-dimensional (3D) object relative to a display surface of an interactive display system is detected based upon the intensity of infrared (IR) light reflected from the object and received by an IR video camera disposed under the display surface. As the object approaches the display surface, a “hover” connected component is defined by pixels in the image produced by the IR video camera that have an intensity greater than a predefined hover threshold and are immediately adjacent to another pixel also having an intensity greater than the hover threshold. When the object contacts the display surface, a “touch” connected component is defined by pixels in the image having an intensity greater than a touch threshold, which is greater than the hover threshold. Connected components determined for an object at different heights above the surface are associated with a common label if their bounding areas overlap.

    Abstract translation: 基于从物体反射并由设置在显示面下方的IR摄像机接收的红外(IR)光的强度来检测三维(3D)物体相对于交互式显示系统的显示面的位置。 当物体靠近显示表面时,由“红外”摄像机产生的图像中的像素由“静止”连接的组件定义,该像素的强度大于预定的悬停阈值,并且紧邻另一像素,其强度也大于 悬停阈值。 当物体接触显示表面时,通过图像中具有大于触摸阈值的强度大于悬停阈值的像素来定义“触摸”连接的分量。 如果其边界区域重叠,则在表面上方不同高度处确定的对象的连接部件与公共标签相关联。

    Method and apparatus for managing text objects for providing text to be
interpreted across computer operating systems using different human
languages
    79.
    发明授权
    Method and apparatus for managing text objects for providing text to be interpreted across computer operating systems using different human languages 失效
    用于管理文本对象的方法和装置,用于提供使用不同人类语言跨计算机操作系统解释的文本

    公开(公告)号:US5802539A

    公开(公告)日:1998-09-01

    申请号:US642059

    申请日:1996-05-03

    Abstract: Text objects having a primary data portion in which is stored text characters and associated encoding information, and an annotation portion in which is stored attribute information such as style and language identifiers, is described. The encoding information is stored within a run header in the primary data portion and both the run header and attribute header refer to the text characters to thereby define a text run. Also described are operations for manipulating the text objects of the invention and for creating and deleting annotations. The operations for manipulating the text objects of the invention include installing text within a text object, copying text in a text object, replacing text in a text object, writing text in a text object and imaging text in a text object for display.

    Abstract translation: 描述具有其中存储有文本字符和相关编码信息的主数据部分的文本对象,以及其中存储有诸如样式和语言标识符的属性信息的注释部分。 编码信息存储在主数据部分中的运行报头中,并且运行报头和属性报头都指的是文本字符,从而定义文本运行。 还描述了用于操纵本发明的文本对象以及创建和删除注释的操作。 用于操作本发明的文本对象的操作包括在文本对象内安装文本,在文本对象中复制文本,替换文本对象中的文本,在文本对象中书写文本以及在文本对象中成像文本以进行显示。

Patent Agency Ranking