In one embodiment, a computing system may generate and display a virtual reality environment to a user. The computing system may determine a head pose of the user based on headset tracking data associated with a headset worn by the user. The computing system may determine a hand pose of the user based on hand tracking data associated with a device held or worn by a hand of the user. The computing system may access scene information associated with the displayed virtual reality environment. The computing system may determine a predicted focal point of the user within the virtual reality environment by processing the head pose, the hand pose, and the scene information using a machine-learning model.