Communication apparatus includes a switch, which includes switching logic, multiple ports for connection to a network, and a management port, and which is configured to assign both a first link-layer address and a second link-layer address to the management port. A host processor includes a memory and a central processing unit (CPU), which is configured to run software implementing a management agent for managing functions of the switch. A network interface controller (NIC) is connected to the management port and is configured to convey incoming management packets, which are directed by the switch to the first link-layer address, to the CPU for processing by the management agent, and to write directly to the memory data contained in incoming remote direct memory access (RDMA) packets, which are directed by the switch to the second link-layer address.