A speaker unit and a portable information terminal that are reduced in size and thickness and still produce a high sound pressure. A speaker unit includes: a yoke having a rectangular outline as seen in a plan view; a magnet member placed on the yoke and magnetized so that an upper surface of the magnet member has an N pole and an S pole aligned in one direction; a coil placed above and spaced from the upper surface of the magnet member; a diaphragm attached to the coil; and a frame supporting the diaphragm. The yoke includes an attachment which protrudes upward from an outer periphery of the yoke for attachment to the frame. The attachment has a recess at least in a central portion of the attachment which crosses the one direction.