A system and method allocates partitions to be used to perform a query to multiple GPUs and each partition is also allocated to a CPU to which the GPU is attached. The GPUs use any assigned data in GPU memory, and otherwise request it from a CPU, which provides it from CPU memory if possible, and otherwise copies it from long term storage, to CPU memory and makes it available to GPU memory. The GPUs process the query against the partitions assigned, optionally by performing portions of the query at a time, for example performing an innermost nested query and then performing the next outermost nested query. The results from any given portion of a query may be accumulated across all GPUs and some or all of the results distributed to each of the GPUs. The remaining work may be balanced by redistributing only some of the accumulated results.