Disclosed herein is a method and system for performing context-based transformation of a video. In an embodiment, a scene descriptor and a textual descriptor are generated for each scene corresponding to the video. Further, an audio context descriptor is generated based on semantic analysis of the textual descriptor. Subsequently, the audio context descriptor and the scene descriptor are correlated to generate a scene context descriptor for each scene. Finally, the video is translated using the scene context descriptor, thereby transforming the video based on context. In some embodiments, the method of present disclosure is capable of automatically changing one or more attributes, such as color of one or more scenes in the video, in response to change in the context of audio/speech signals corresponding to the video. Thus, the present method helps in effective rendering of a video to users.