An approach for de-fragmenting physical memory generally involves grouping kernel pages together based on large pages. The de-fragmentation procedure is triggered, such as by a kernel page-freelist being empty. The first user page from a user page-freelist is selected, marked as a kernel page (e.g., by setting a P_KERNEL bit), added to the kernel page-freelist, and then the large page in which the selected page is identified. Starting with the first small page within the large page, the small pages are processed by the de-fragmentation procedure, resulting in as many small pages as possible being marked as kernel pages and then added to the kernel page-freelist. Later, when a large page is coalesced, the number of kernel pages that must be relocated within the large page being coalesced is reduced or eliminated as a result of the de-fragmentation of the physical memory.