Described herein are various techniques and principles for determining how to assign control of peripherals and assigning control of peripherals. In some embodiments, determining how to assign control of peripherals comprises reviewing connections of peripherals to the computing device and evaluating rules to determine management points in the connections. In some cases, the connections of peripherals to the computing device may be organized into a hierarchy corresponding to a hierarchy of physical connections of the peripherals, including physical connections of peripherals located remote from the computing device and possibly connected through another computing device. When management points are identified among the connections, control of peripherals associated with the management points may be assigned in the same way. For example, access rights to each of the peripherals may be assigned to a same user session.