A system and method for cluster topology aware container scheduling for efficient data transfer. The method begins by configuring a server architecture that includes an inner ring of servers including a server that is a specialized hardware accelerator that provides a specialized computation, an outer ring of servers that includes central processing unit servers and no specialized hardware accelerator, and a network switch to transfer data between the inner ring and the outer ring of servers. An orchestrator receives a request to execute a container, and the orchestrator reads a latency tolerance of the container, where the latency tolerance can be either a real time or a batch tolerance. The scheduler schedules the container to execute, based on the latency tolerance of the container, in either the inner ring or the outer ring of servers. The orchestrator injects the container with instructions to access the specialized hardware accelerator.