Techniques are described for controlling operation of both a host device and a wearable display device connected to the host device based on a use status of the wearable display device. The techniques include automatically determining a use status of a wearable display device based on feedback from one or more touch sensors within the wearable display device that indicates whether the wearable display device is worn by a user. Based on the determined use status, the wearable display device controls its own operation (e.g., controls operation of display screens of the wearable display device, a communication session with the host device, and display processing of data received from the host device). The wearable display device also sends an indication of the use status to the host device. The host device then controls its own data processing for the wearable display device based on the indicated use status.