Techniques for optimizing virtual machine (VM) scheduling on a non-uniform cache access (NUCA) system are provided. In one set of embodiments, a hypervisor of the NUCA system can partition the virtual CPUs of each VM running on the system into logical constructs referred to as last level cache (LLC) groups, where each LLC group is sized to match (or at least not exceed) the LLC domain size of the system. The hypervisor can then place/load balance the virtual CPUs of each VM on the system's cores in a manner that attempts to keep virtual CPUs which are part of the same LLC group within the same LLC domain, subject to various factors such as compute load, cache contention, and so on.