A paper picking-up unit and an image forming apparatus having the same are provided. The paper picking-up unit includes a knock-up plate to stack paper thereon; a shaft disposed at a front end of the knock-up plate to rotate, a pick-up roller engaged with the shaft to rotate and pick-up the paper stacked on the knock-up plate, and a knock-up plate ascending and descending portion to raise and lower the knock-up plate according to the rotation of the pick-up roller. The pick-up roller includes a roller body engaged and rotatable with the shaft, and having a frictional layer to pick-up the paper by causing friction with the paper, a pair of roller housings to support opposite sides of the roller body such that the roller body does not move in a lengthwise direction of the shaft, and a damper disposed at an outer circumference of each roller housing to reduce noise caused by a collision of the roller housings and the knock-up plate.