Embodiments of the disclosure relate to verifying a watermark of an artificial intelligence (AI) model for a data processing (DP) accelerator. In one embodiment, a system receives an inference request from an application. The system extracts the watermark from an AI model having the watermark. The system verifies the extracted watermark based on a policy. The system applies the AI model having a watermark to a set of inference inputs to generate inference results. The system sends a verification proof and the inference results to the application.