Embodiments are directed to security optimizing compute distribution in a hybrid deep learning environment. An embodiment of an apparatus includes one or more processors to determine security capabilities and compute capabilities of a client machine requesting to use a machine learning (ML) model hosted by the apparatus; determine, based on the security capabilities and based on exposure criteria of the ML model, that one or more layers of the ML model can be offloaded to the client machine for processing; define, based on the compute capabilities of the client machine, a split level of the one or more layers of the ML model for partition of the ML model, the partition comprising offload layers of the one or more layers of the ML model to be processed at the client machine; and cause the offload layers of the ML model to be downloaded to the client machine.