An apparatus and a method. The apparatus includes a first recurrent network in a first layer; a second recurrent network in a second layer connected to the first recurrent network; a distant input gate connected to the second recurrent network; a first highway gate connected to the distant input gate and the second recurrent network; a first elementwise product projection gate connected to the distant input gate, the highway gate, and the second recurrent network; a second highway gate connected to the first recurrent network and the second recurrent network; and a second elementwise product projection gate connected to the first recurrent network, the second highway gate, and the second recurrent network.