A method of processing a signal is disclosed, which includes receiving a downmix signal, object information and rendering information; obtaining downmix gain flag information indicating whether a downmix gain is included in the object information; and generating downmix processing information using the object information and the rendering information based on the downmix gain flag information, wherein the object information includes modified object level information or object level information, wherein the modified object level information is determined using a level of a modified object generated by applying the downmix gain to an object, and wherein the object level information is determined using a level of the object. Accordingly, in an apparatus for processing a signal and method thereof according to the present invention, the signal is decoded using modified object level information based on downmix gain flag information. Therefore, the present invention enables to select an efficient signal processing method according to an environment.