System and method for speech-based navigation and interaction with a device's visible screen elements using a corresponding view hierarchy
Abstract:
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for enabling screen-specific user interfacing with elements of viewable screens presented by an electronic device are disclosed. In one aspect, a method includes the actions of identifying a character sequence representing a first input that is received while displaying a viewable screen having at least one selectable viewable element. The actions further include accessing an electronic file that provides a text representation of one or more of the at least one selectable viewable element. The actions further include comparing the character sequence to the text representation. The actions further include selecting, within the viewable screen, a selectable viewable element whose text representation matches the character sequence. The actions further include triggering any action linked to the selecting the selectable viewable element.
Information query
Patent Agency Ranking
0/0