Sophisticated memory systems and intelligent machines may be constructed by creating an active memory system with a hierarchical architecture. Specifically, a system may comprise a plurality of individual cortical processing units arranged into a hierarchical structure. Each individual cortical processing unit receives a sequence of patterns as input. Each cortical processing unit processes the received input sequence of patterns using a memory containing previously encountered sequences with structure and outputs another pattern. As several input sequences are processed by a cortical processing unit, it will therefore generate a sequence of patterns on its output. The sequence of patterns on its output may be passed as an input to one or more cortical processing units in next higher layer of the hierarchy. A lowest layer of cortical processing units may receive sensory input from the outside world. The sensory input also comprises a sequence of patterns.