In one general aspect, systems and methods are described for determining positional data for three-dimensional interactions inside virtual reality environments. The methods may include receiving inertial measurement unit (IMU) data from at least one mobile device of a plurality of mobile devices operating in a virtual reality environment, obtaining absolute position and orientation data from at least one tracked controller, comparing the IMU data from the at least one mobile device with the absolute position and orientation data of the at least one tracked controller, detecting a correlation between the IMU data from the at least one mobile device and the absolute position and orientation data of the at least one tracked controller. Based on the detected correlation, the method may include determining absolute position and orientation for the at least one mobile device based on a determined relative orientation for the at least one tracked controller.