A system to support validation and debugging of compiled low-level instructions for a machine learning (ML) network model on an ML-specific hardware. A compiler identifies well-defined boundaries in the ML network model based on primitives used to generate low-level instructions for the hardware. The ML network model is partitioned into units/layers/sub-graphs based on the plurality of well-defined boundaries. The compiler then generates an internal representation for each of the units wherein the internal representation is mapped to components in the hardware. Each of the units is compiled into a first set to be executed on the ML-specific hardware and a second set to be executed on a second computing device. The output results from executing the two sets of low-level instructions are compared to validate the first set of low-level instructions. If the outputs do not match fully, the first set of low-level instructions is debugged and recompiled.