A compact electromagnetic actuator for driving a lens includes a lens holder used to hold a lens or a lens assembly which defines an optical axis, an image stabilization (IS) actuator provided at one end of the lens holder for producing a linear motion in a plane perpendicular to the optical axis, and an autofocus (AF) actuator provided at an opposite end of the lens holder for producing a linear motion along the optical axis. An AF connection mechanism can be used to connect the autofocus actuator to the lens holder. An IS connection mechanism can be used to connect the image stabilization actuator to the lens holder.