A flow-based network switching system includes a memory having a flow table and a packet processor coupled to the memory. The packet processor includes a user-programmable flow-based rule storage that includes a plurality of flow-based rules. A flow-based handler and session manager in the packet processor is operable to retrieve application layer metadata from a first packet received over a network, determine a first flow session associated with the first packet using the application layer metadata from the first packet and the flow table, and retrieve at least one of the plurality of flow-based rules from the programmable flow-based rule storage using the application layer metadata from the first packet. A flow-based rule processing engine in the packet processor is operable to apply the at least one flow-based rule to the first packet. Packets with applied flow-based rules are forwarded through the network.