A method, a system and a computer program product for supporting a reconfigurable hardware network topology including graphics processor units (GPU) and central processing unit (CPU) interconnectivity. The re-configurability of the network is based on data bandwidth and latency requirements of running workloads obtained by running training workload sequences upon the configured topology. For network re-configurability, a user enabled to directly connect high-speed cable links between CPU/GPU connectors and between GPU/GPU connectors. Further included in a configured network topology are switches configurable to provide one or more PCIe high-speed side-band links, as well as a high speed multiplexor, that can switch the network topology. The method prepares a system map of the hardware network topology, and generates a GPU weightage pattern based on data bandwidth and latencies of the configured GPUs such that GPUs are assigned to workloads based on its weightage to optimize workload performance.