Topic: Search indexing

Well, today I was looking through google and I noticed none of my forum pages were getting indexed. I decided this was obvioulsy because I forced all users to login in order to view the pages. I know that it kinda defeats the purpose of forcing uses to login, but still have them be able to view the forums by viewing cached google pages. Anyways I threw together a quick hack

    $agent = $_SERVER['HTTP_USER_AGENT'];
    if (strstr(strtolower($agent), "mozilla") == false)
        $pun_user['g_read_board'] = 1;

for the default user function, and yes I know it sucks. Basicly I am wondering if anyone knows of a better way to only allow spiders to view the forums, but still disallow users

I enjoy pie :)

Re: Search indexing

This isn't complete safe, because people can change the user agent field in their HTTP headers, and pretend to be the Googlebot. I don't think there's a safe way to do this.

Re: Search indexing

then again, if its safe for google to index your site why don't you want people to see it?

Re: Search indexing

We figure if you are registered you are more likely to post. So if you can just read the forums as guests and never plan on registering and then posting and contributing, why waste our bandwidth?

I enjoy pie :)

Re: Search indexing

I'm... without a voice anymore...

6

Re: Search indexing

Something to do with robots.txt . Can't remeber what though.

Do, or do not.

Re: Search indexing

you can restrict pages with robots.txt not circumvent php stuff