In one embodiment, a method includes receiving, from a client device of an author of a message, a request for a restricted ideogram to be inserted into a message; accessing social-networking information for the author; determining, based on the social-networking information for the author, whether the author is authorized to access the restricted ideogram; accessing social-networking information for a recipient user; determining, based on the social-networking information for the recipient user, whether the recipient user is authorized to access the restricted ideogram; and if the author and the recipient user are authorized to access the restricted ideogram, then sending, to the client device of the author, information to insert the restricted ideogram into the message.