A method and apparatus for enabling an external application such as a web crawler access to dynamic web pages associated with a primary application such as a portal page. The primary application addresses each component associated with it and requests a list of resource identifiers. Each component implements an interface and provides such list of resource identifiers. The list is returned to the external application, which then optionally requests the contents of the page associated with each resource identifier. The component provides the content of the page, which is then parsed by a parsing module associated with the primary application. The parsing module transforms the content into a data structure such as a Document Object Model, and then extracts text or Hypertext Markup Language code from the data structure. The text is then returned to the external application fro searching, indexing or other purposes.