Techniques of generating a virtual environment in a virtual reality system involves changing, within a user interface of the second user, an attribute of an avatar representing the first user while maintaining a spatial position of an object with which the first user is interacting. In this way, the second user may see only non-threatening or otherwise pleasant avatars within their user interface while other users may not perceive any change to the virtual environment as displayed in their respective user interfaces.