An interactive streaming media and application service provider system can securely stream high resolution, multiple formats of video and data. Different data sets can be included in a single stream. A rights management system controls matrix manipulation and other aspects of user control of the data, including one or more of rendering in various different 2D, 3D, or other media formats, reconstruction and modeling, zooming, frame grab, print frame, parental controls, picture in picture, preventing unauthorized copying, adapting to different data transmission formats, adapting to different resolutions and screen sizes, and actively control functionality contained in embedded data, encryption/decryption. Control can be exerted by an external entity through a user-side virtual machine. Control codes can optionally be embedded in the media, embedded in the user's device, and/or sent separately to the device.