A URL processing method includes a response data determining step in which a URL processing apparatus determines whether to exclude one or more URLs included in a first web page from a valid URL list using header information for the first web page of a first web site and a similarity based valid URL calculating step of estimating a similarity between web pages corresponding to respective URLs according to a predetermined criterion with respect to one or more URLs included in the first web page and selecting some of URLs of a similar web page calculated according to the similarity and adding the selected URLs in the valid URL list.