In a computer network environment, a content-based session request issued by a search engine agent is modified to enable the search engine to index information from a server to maintain a state connection where the search engine agent cannot maintain session identification. The content-based session request is modified by a header intermediary module which then directs the request to target servers. The response from the web server is again modified by the header intermediary module prior to being sent to the client that issued the request. It is emphasized that this abstract is provided to comply with the rules requiring an abstract which will allow a searcher or other reader to quickly ascertain the subject matter of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or the meaning of the claims.