System, method, and machine readable medium implementing a mechanism for selecting and providing reconfigurable hardware resources in a rack architecture system are described herein. One embodiment of a system includes a plurality of nodes and a configuration manager. Each of the nodes further includes: a plurality of memory resources and a node manager. The node manager is to track the memory resources that are available in the node, determine different possible configurations of memory resources, and generate a performance estimate for each of the possible configurations. The configuration manager is to receive a request to select one or more nodes based on a set of performance requirements, receive from each node the different possible configurations of memory resources and the performance estimate for each of the possible configurations, and iterate through collected configurations and performance estimates to determine one or more node configurations best matching the set of performance requirements.