Systems and methods are described herein for selecting a closed captioning language track. Control circuitry may capture an image or video of a user while the user is performing a gesture of a sign language. The control circuitry may access a database comprising a plurality of entries corresponding to a plurality of gestures in respective sign languages. The control circuitry may compare a first pixel map corresponding to the image or video of the user with pixel maps corresponding to each of the plurality of entries in order to identify a preferred sign language. The control circuitry may receive metadata associated with a media asset comprising a plurality of closed captioning language tracks, each closed captioning language track comprising closed captioning for the media asset in a different language and select one of the closed captioning language tracks that corresponds to the preferred sign language.