A method and a system for maintaining network integrity for incrementally training machine learning (ML) models at edge devices is provided. The method includes registering, by a certifying node, one or more edge devices with a peer to peer network. Upon registration, an incrementally updated ML model is received from a first registered device at the certifying node. The certifying node accepts the incrementally updated ML model if a contribution of the first edge device is within a predetermined threshold, and else rejects the updated ML model if the contribution is beyond the predetermined threshold. Limiting the contribution by each edge device enables prevention of skew by any of the edge devices at the certifying node. Upon accepting the updated ML model, the certifying node certifies the updated ML model and transfers the certified ML model to one or more other edge devices in the peer to peer network.