A virtual conferencing system includes a 3D scanner in a local region and a camera. A scan of the local region and images are received and combined to make a model that is transmitted to a remote region. A remote model from the remote region is combined and rendered for a user in the local region. A reference point in the local region is mapped to a reference point in the remote region. The remote model is oriented with respect to the local region according to the reference points. Object in the remote and local regions may be filtered from models that are transmitted. Interactions with virtual objects in the local region may invoke modification virtual objects. Modifications may be transmitted to the remote region and superimposed on real objects corresponding to the virtual objects.