Technologies for detecting interactions with surfaces from a spherical view of a room include a compute device. The compute device includes an image capture manager to obtain one or more images that depict a spherical view of a room that includes multiple surfaces. Additionally, the compute device includes a surface interaction detection manager to detect, from the one or more images, a person in the room, generate a bounding box around the person, preprocess the bounding box to represent the person in an upright orientation, determine a pose of the person from the preprocessed bounding box, detect an outstretched hand from the determined pose, and determine, from the detected outstretched hand, a surface of interaction in the room.