1 (edited by Programming-Designs 2005-11-22 22:46)

Topic: GoogleBot

I saw another forum that would show whenever GoogleBot was visiting their forum, so I was wondering if anybody could write a lil mod that would show "GoogleBot" in the user's online list whenever it comes around.

I know there isn't really any point of this, but I think it'd be a cool little feature to add smile

I found out how to detect googlebot, with this code snippet:

if(strstr(strtolower($HTTP_USER_AGENT), "googlebot")){
        do_work_here;
}

How would I include him into the "online users" list?

Re: GoogleBot

just modify your index.php

at line 179 youll find this
        if ($cur_user_online['user_id'] > 0)

I can only think it'd have something to do with that.

And I agree, this'd be very cool, although I'd like to add support for the MSNbot etc also (which I can only assume wouldn't be much harder to do).

3 (edited by af3 2006-10-26 16:05)

Re: GoogleBot

how abt this in in index.php

    if ($num_users > 0) {
        if(strstr(strtolower($HTTP_USER_AGENT), "googlebot")){ echo "\t\t\t".'<dl id="onlinelist" class= "clearb">'."\n\t\t\t\t".'<dt><strong>'.$lang_index['Online'].': </strong></dt>'."\t\t\t\t".implode(',</dd> ', $users).' GoogleBot</dd>'."\n\t\t\t".'</dl>'."\n";}
        else { echo "\t\t\t".'<dl id="onlinelist" class= "clearb">'."\n\t\t\t\t".'<dt><strong>'.$lang_index['Online'].': </strong></dt>'."\t\t\t\t".implode(',</dd> ', $users).'</dd>'."\n\t\t\t".'</dl>'."\n";}
    }
    else
        echo "\t\t\t".'<div class="clearer"></div>'."\n";

4 (edited by Programming-Designs 2005-11-23 05:09)

Re: GoogleBot

af3, are you sure that will work? I thought that would only show googlebot on for google bot wouldn't it? Because it's seeing if the user agent is googlebot, so if it isn't google bot nobody else will see google bot listed as online would they?

I'm pretty sure there is more to it than that.

Re: GoogleBot

Yeh Programming-Designs is right, its not an easy thing to do.

6

Re: GoogleBot

Guys -- i was asking if the code would work, obviously not at all. just a thought: since google bot comes from an ip, can we check the IP and use this somehow to detect googlebot presence? i dont know how though.

when a user visit the page, the set_default_user() in function.php is called. can we put some code like this within that to update the online users lists?

    $googlespiders = array("Froogle", "Googlebot");
    foreach($googlespiders as $onlinebot) {
      if(ereg($onlinebot, $HTTP_USER_AGENT)) {
          if ($onlinebot == "Googlebot") {
            if (substr($REMOTE_HOST, 0, 11) == "209.185.253.") $onlinebot = "Googlebot Deep Crawl";
            elseif (substr($REMOTE_HOST, 0,7) == "64.209.") $onlinebot = "Google Crawler";
// then try to insert to database
            mysql_query ("INSERT INTO bots (name) VALUES ('$onlinebot')");
          }
      }
  }

ps: i'm not a php coder. just an idea.

Re: GoogleBot

af3: if you're using set_default_user then you can use useragent as well wink

Re: GoogleBot

Connorhd, do you know how I would go about getting this to work? I'd be interesting in writing the mod myself if somebody could point me to the right tables/information to access/modify for it to work.

Thanks.

Re: GoogleBot

actually what af3 said seems like the best idea, not excatly sure how to implement it, and sorry but i don't really want to either, seems like a waste of time to me tongue

10 (edited by Programming-Designs 2005-11-24 16:36)

Re: GoogleBot

Connorhd, I actually found a simple solution to this. Instead of using ANY database connectivity I keep a file that stores information about the Bot's Display Name, the Bot's Identity (HTTP_USER_AGENT), and the timestamp for the bot's last visit. Has delimiters and such for multiple bots. I just have it so that it displays the bot's name if it has visited in the last 5 minutes.

So all I had to do was include the php file i wrote that does all that, and determine what I want to do if it returns data when I call the function. smile

EDIT:
If you're interested in seeing in action, first go here: http://www.smart-it-consulting.com/inte … /index.htm (a bot spoofer)

Enter the url http://programming-designs.com/forums/index.php and choose either Google, MSN, Yahoo, Alexa-1, or the Yahoo-Slurp bot.

After you spoofed the visit, go to my forums and look at users online: http://programming-designs.com/forums

Re: GoogleBot

but a file will make it rather slow won't it?

Re: GoogleBot

^ I can't notice a slow down really, but until I learn more about PunBB and creating tables/modifying them this will have to do xD

Re: GoogleBot

Can we have a writeup then? I'm interested on potentially displaying this on my site, because it seems handy.

Re: GoogleBot

^ Will do, busy at the moment, but will probably write one tomorrow. Then if anybody is interested they can choose to re-write it if they'd like using the PunBB sql database.