An image processing apparatus enables image data to be handled easily after photography. A server apparatus patterns images and controls information in correspondence with user identifiers. A digital camera photographs a subject, obtains a pattern image and a piece of control information from the server apparatus, and judges whether a portion of the image data obtained by photography matches the obtained pattern image. If a matching portion exists, then the digital camera operates in accordance with an operation instruction included in the piece of control information. If the operation instruction is mosaic processing, then the digital camera performs mosaic processing to the matching portion. If the operation instruction is image replacement, then the digital camera replaces the matching portion with a predetermined image or performs CG processing on the matching portion. If the operation instruction is frame deletion, then the digital camera deletes the image data.