Provided is a method and apparatus for decreasing a handover latency time using a context exchange between a mobile terminal (MT) and a Media Independent Handover Server (MIHS). The method used by the MT for decreasing handover latency between a first network and a second network includes when a handover is performed from the first network to the second network, transmitting to the MIHS information required for first network communication as context; and when a handover is performed from the second network to the first network, receiving from the MIHS first network access information comprising the context through the currently used second network. Accordingly, the MT can reduce a Vertical Handover (VHO) latency time and simultaneously consume almost the same power as a single mode terminal.