Sensor fusion algorithm techniques are described. In one or more embodiments, behaviors of a host device and accessory devices are controlled based upon an orientation of the host device and accessory devices, relative to one another. A combined spatial position and/or orientation for the host device may be obtained based on raw measurements that are obtained from at least two different types of sensors. In addition, a spatial position and/or orientation for an accessory device is ascertained using one or more sensors of the accessory device. An orientation (or position) of the accessory device relative to the host computing device may then be computed based on the combined spatial position/orientation for the host computing device and the ascertained spatial position/orientation for the accessory device. The relative orientation that is computed may then be used in various ways to control behaviors of the host computing device and/or accessory device.