The disclosed system incorporates a new learning module, the Learning Kernel Activation Module (LKAM), at least serving the purpose of enforcing the utilization of less convolutional kernels by learning kernel activation rules and by actually controlling the engagement of various computing elements: The exemplary module activates/deactivates a sub-set of filtering kernels, groups of kernels, or groups of full connected neurons, during the inference phase, on-the-fly for every input image depending on the input image content and the learned activation rules.