Techniques for providing a unified computational platform for implementing virtual reality systems. In an aspect, a virtual reality (VR) client includes input and output modules for generating a VR user experience. The client may also incorporate an assistant interface that couples the input and output modules to an assistant server using a semantic information interchange language (SIIL). The assistant server may include a command inference block for converting user-input SIIL messages to command-input SIIL messages, thereby encoding user VR inputs into a general command set that may be recognized and processed by both VR and non-VR modules. Further aspects disclose techniques for seamless integration between virtual environments and joint VR sessions supporting multiple users.