• Registered: 2007-01-26
  • Posts: 11

Topic: Restrict bots to some sections

Hello
How it would be possible to do so that the bots foot are automatically assimilated with a determined group. While leaving all forums opened to the normal guests I would like to restrict the rights of access of Googlebot, Msnbot and all others bots in certain forums. My idea was to create a group with restrained rights and to be able to link it the bots a bit as principle of  " How to make year undercover Admin ".
I searched in the forum and Iask me if it would not be possible to use the botdetect.php of " BotDetect Mod Release v1.5 " ?
Thanks a lot.

  • Smartys
  • Former PunBB Developer
  • Offline
  • Registered: 2004-04-30
  • Posts: 7,844

Re: Restrict bots to some sections

Moved to Modifications
Why not just use robots.txt?

  • Registered: 2007-01-26
  • Posts: 11

Re: Restrict bots to some sections

It is not possible with robots.txt who allows or not the access to files or directories. With a group it is possible to modulate permissions to see or not different posts and forums.

  • Smartys
  • Former PunBB Developer
  • Offline
  • Registered: 2004-04-30
  • Posts: 7,844

Re: Restrict bots to some sections

Fair enough wink
Out of curiosity, why would you want to let regular guests access some forums but not let Google? I would think that it would be to your benefit to let Google read the forums that you let your guests read wink

  • Registered: 2007-01-26
  • Posts: 11

Re: Restrict bots to some sections

It is however simple. Bots are useful but very curious.  I do not wish that Google keeps in its cache the data of userlist, profiles of members and contents of posts. By allocating the bots to a special group they could categorize only the url and names of forums and titles of posts without having access to contents. The guests who visit Google or others seach engines and follow they pages have them full access to the forum but nothing is to keep in cache.

  • Smartys
  • Former PunBB Developer
  • Offline
  • Registered: 2004-04-30
  • Posts: 7,844

Re: Restrict bots to some sections

It is however simple. Bots are useful but very curious.  I do not wish that Google keeps in its cache the data of userlist, profiles of members and contents of posts.

So, you would serve up different versions of the page to the search bots? Search engines generally don't like that: if you just don't want Google, etc to cache the contents of your pages, use the noarchive meta tag