An apparatus in one embodiment comprises a plurality of container host devices of at least one processing platform. The container host devices implement a plurality of containers for executing applications on behalf of one or more tenants of cloud infrastructure. The containers are separated into at least first and second distinct container groups based at least in part on results of comparisons of layer structures each characterizing container images of respective different ones of the containers. At least one grouping-based container management action is determined for at least one of the containers based at least in part on which of the container groups includes that container. The grouping-based container management action is applied to one or more of the containers of one of the first and second container groups but not to the containers of the other of the first and second container groups.