Techniques for using motion recognizers are described. The motion recognizers are created or generated in advance by trained users. The motion recognizers are then loaded into a processing unit that receives motion signals from one or more motion sensitive devices being manipulated by one or more end users to control one or more objects in a virtual environment. Depending on implementation, the virtual environment may represent a remote scene or a video game, where objects in the virtual environment can be controlled by the users to perform desired actions or moves.