Systems and methods are described for providing connection pools between source network devices and a target, connection-limited service. Each connection pool can facilitate connections between source devices and the target service, while ensuring that connections to the connection-limited service do not exceed a defined limit. A connection manager service can initialize a connection pool for a target service on request by a client device, and provide an identifier for the connection pool to the client device. Source network devices can then transmit operations for the target service to the connection manager service, which can route the operations to an appropriate connection pool based on the identifier.