One embodiment of the present invention provides a computing system. The computing system includes a processor and a computer-readable storage medium for storing instructions. Based on the instructions, the processor operates the computing system as an overlay gateway. The computing system initiates and terminates an overlay tunnel associated with a virtual machine. During operation, the computing system maps a virtual Internet Protocol (IP) address of the virtual machine to a second IP address used to terminate the overlay tunnel based on information received from a configuration system. The computing system then determines an output port for a data packet based on the second IP address. The data packet comprises an inner packet and the destination address of this inner packet corresponds to the virtual IP address.