Aggregate statistics are determined by first randomizing independently data X and Y to obtain randomized data {circumflex over (X)} and Ŷ. The first randomizing preserves the privacy of the data X and Y. Then, the randomized data {circumflex over (X)} and Ŷ is randomized secondly to obtain randomized data {tilde over (X)} and {tilde over (Y)} for a server, and helper information T{tilde over (X)}|{circumflex over (X)} and TŶ|Ŷ for a client, wherein T represents an empirical distribution, and wherein the randomizing secondly preserves the privacy of the aggregate statistics of the data X and Y. The server then determines T{tilde over (X)},{tilde over (Y)}. Last, the client applies the side information T{tilde over (X)}|{circumflex over (X)} and TŶ|Ŷ to T{tilde over (X)},{tilde over (Y)} to obtain an estimated {dot over (T)}X,Y, where “|” and “,” between X and Y represent a conditional and joint distribution, respectively.