Topic: Google Sitemaps

Now that Google has released an XML specification for keeping track of site updates it'd be cool if punbb could implement an auto-updating xml file that could be submitted to Google Sitemaps and any other service that eventually takes part in the open specification.

https://www.google.com/webmasters/sitemaps/

Re: Google Sitemaps

Smartys is working on one wink (well i think he might have finished actually)

Re: Google Sitemaps

I did, although I actually do want to add something
I'll release it today I guess wink

Re: Google Sitemaps

http://www.punres.org/files.php?pid=90

Re: Google Sitemaps

Awesome. I've got it installed at http://theforum.tv/sitemap.php and it looks like it works perfectly. I submitted it to Google Sitemaps, but it's currently pending. (I hope they don't do a human review of every sitemap that gets submitted wink)

Re: Google Sitemaps

Cool smile
I'm probably still early, but does it work?

7 (edited by adietler 2005-06-09 01:40)

Re: Google Sitemaps

Yeap. The Google Sitemaps site said it was downloaded 8 hours ago and its status is "OK" and there are no errors on the errors page, so I assume it's doing it's job.

Thanks for making this Smartys.

Re: Google Sitemaps

No problem smile

Re: Google Sitemaps

Won't that give lastmod times in the future for servers running in the eastern hemisphere since you use the local time, but claim it to be GMT ?

Also it should be H:i:s, not G:i:s. smile

Re: Google Sitemaps

Just checked the specs they want to hold people to, and it does need the leading 0 that H provides smile

Updated, and I'll upload a new version (with the ability to output a static page) later today

Re: Google Sitemaps

Why is viewforum.php?id=12 for example in there twice?

Re: Google Sitemaps

It isn't wink

Re: Google Sitemaps

Smartys wrote:

Just checked the specs they want to hold people to, and it does need the leading 0 that H provides smile

Updated, and I'll upload a new version (with the ability to output a static page) later today

Did you ever complete this new version?

14 (edited by Smartys 2005-06-15 15:29)

Re: Google Sitemaps

I could have sworn I did, one sec
Edit: Here it is

15

Re: Google Sitemaps

Wow, thats sweet! wink
Been too busy to look into this much, been hearing a lot of buzz lately.

That gives new meaning to the term "EZ install" too big_smile

Thanx a bunch

16 (edited by gog 2006-06-23 17:16)

Re: Google Sitemaps

There is one problem with this mod. This mod only provides link to the first page of the topic. What about subsequent pages. I guess this  mod should be rewriten to check the configuration info on posts per page setting, count the number of posts in each thread, and than create additional links...

http://www.info-mob.com/forum/ - Croatian forum only, don't bother if you don't speak Croatian :)

Re: Google Sitemaps

My thought at the time was that all the other pages would be subsequently linked through the navigation on the bottom wink
I happen to be going away for 3 weeks starting tomorrow, so I'll set what I can do fast tongue

18

Re: Google Sitemaps

All pages are linked from forums index.php if you look at it that way smile

If you don't do it, I'll do it, just need to get some spare time...

http://www.info-mob.com/forum/ - Croatian forum only, don't bother if you don't speak Croatian :)

19 (edited by Smartys 2006-06-23 17:55)

Re: Google Sitemaps

All pages are linked from forums index.php if you look at it that way

True tongue

Here's a modified version, tell me if it works OK, I'll release it if it does

<?php
/***************************************************************************
 *                              googlesitemapgenerator.php
 *                            -------------------
 *   Copyright/Support          http://www.pentapenguin.com
 *   Last Modified: 06/05/05
 *     Modified by Smartys for use with PunBB
 *
 ***************************************************************************/
 
/***************************************************************************
 *
 *   This program is free software; you can redistribute it and/or modify
 *   it under the terms of the GNU General Public License as published by
 *   the Free Software Foundation; either version 2 of the License, or
 *   (at your option) any later version.
 *
 ***************************************************************************/

define('PUN_QUIET_VISIT', 1);
define('PUN_ROOT', './');
require PUN_ROOT.'include/common.php';

// false = write to file, true = dynamic
$dynamic = true;

// This only matters if you're writing to the file
$filename = 'sitemap.xml';


// Get the topics
$result = $db->query('SELECT t.id as topic_id, last_post, sticky, num_replies FROM '.$db->prefix.'topics AS t LEFT JOIN '.$db->prefix.'forum_perms AS fp ON (fp.forum_id=t.forum_id AND fp.group_id=3) WHERE (fp.read_forum IS NULL OR fp.read_forum=1) AND t.moved_to IS NULL ORDER BY last_post DESC') or error('Unable to fetch topic list', __FILE__, __LINE__, $db->error());

// Get the forums
$result2 = $db->query('SELECT f.id as forum_id, last_post, num_topics FROM '.$db->prefix.'forums AS f LEFT JOIN '.$db->prefix.'forum_perms AS fp ON (fp.forum_id=f.id AND fp.group_id=3) WHERE fp.read_forum IS NULL OR fp.read_forum=1 ORDER BY f.id DESC') or error('Unable to fetch forum list', __FILE__, __LINE__, $db->error());


$output = '<xml version="1.0" encoding="UTF-8"?>'."\n";
$output .= '<urlset xmlns="http://www.google.com/schemas/sitemap/0.84">'."\n";

// The board itself
$output .= "<url>\n";
$output .= "\t<loc>".$pun_config['o_base_url']."/</loc>\n";
$output .= "\t<lastmod>".gmdate('Y-m-d\TH:i:s+00:00', time())."</lastmod>\n";
$output .= "\t<priority>1.0</priority>\n";
$output .= "</url>\n\n";


// Output the data for the forums
while ($cur_forum = $db->fetch_assoc($result2))
{
    $lastmodified = gmdate('Y-m-d\TH:i:s+00:00', $cur_forum['last_post']);
    $viewforum = 'viewforum.php?id='.$cur_forum['forum_id'];
    $priority = '1.0';
    
    if ($cur_forum['num_topics'] >= $pun_config['o_disp_topics_default'])
    {
        $num_pages = ceil($cur_forum['num_topics'] / $pun_config['o_disp_topics_default']);
        
        for ($i = 1; $i <= $num_pages; $i++)
        {
            $output .= "<url>\n";
            $output .= "\t<loc>".$pun_config['o_base_url']."/".$viewforum."&p=".$i."</loc>\n";
            $output .= "\t<lastmod>$lastmodified</lastmod>\n";
            $output .= "\t<priority>$priority</priority>\n";
            $output .= "</url>\n\n";
        }
    }
    else
    {
        $output .= "<url>\n";
        $output .= "\t<loc>".$pun_config['o_base_url']."/".$viewforum."</loc>\n";
        $output .= "\t<lastmod>$lastmodified</lastmod>\n";
        $output .= "\t<priority>$priority</priority>\n";
        $output .= "</url>\n\n";
    }
}

// Output the data for the topics
while ($cur_topic = $db->fetch_assoc($result))
{
    $lastmodified = gmdate('Y-m-d\TH:i:s+00:00', $cur_topic['last_post']);
    $viewtopic = 'viewtopic.php?id='.$cur_topic['topic_id'];
    $priority = ($cur_topic['sticky'] == '1') ? '1.0' : '0.5';
    
    if ($cur_topic['num_replies'] >= $pun_config['o_disp_posts_default'])
    {
        // We add one because the first post is not counted as a reply but needs to be
        // taken into account for display
        $num_pages = ceil(($cur_topic['num_replies'] + 1) / $pun_config['o_disp_posts_default']);
        
        for ($i = 1; $i <= $num_pages; $i++)
        {
            $output .= "<url>\n";
            $output .= "\t<loc>".$pun_config['o_base_url']."/".$viewtopic."&p=".$i."</loc>\n";
            $output .= "\t<lastmod>$lastmodified</lastmod>\n";
            $output .= "\t<priority>$priority</priority>\n";
            $output .= "</url>\n\n";
        }
    }
    else
    {
        $output .= "<url>\n";
        $output .= "\t<loc>".$pun_config['o_base_url']."/".$viewtopic."</loc>\n";
        $output .= "\t<lastmod>$lastmodified</lastmod>\n";
        $output .= "\t<priority>$priority</priority>\n";
        $output .= "</url>\n\n";
    }
}
$output .= "</urlset>\n";

// If we chose dynamic, we output the sitemap
// Otherwise, we write it to the file
if ($dynamic)
{
    header('Content-type: application/xml');
    echo $output;
}
else
{
    $file = fopen($filename, "w");
    fwrite($file, $output);
    fclose($file);
    echo "Done";
}
?>

20

Re: Google Sitemaps

Yuu are missing ? befor xml in:

$output = '<xml version="1.0" encoding="UTF-8"?>'."\n";

I also edited a code a little bit, so that the first page in the thread doesn't get "&p=1", this is what I ended with:

<?php
/***************************************************************************
 *                              googlesitemapgenerator.php
 *                            -------------------
 *   Copyright/Support          http://www.pentapenguin.com
 *   Last Modified: 06/05/05
 *     Modified by Smartys for use with PunBB
 *
 ***************************************************************************/

/***************************************************************************
 *
 *   This program is free software; you can redistribute it and/or modify
 *   it under the terms of the GNU General Public License as published by
 *   the Free Software Foundation; either version 2 of the License, or
 *   (at your option) any later version.
 *
 ***************************************************************************/

define('PUN_QUIET_VISIT', 1);
define('PUN_ROOT', './');
require PUN_ROOT.'include/common.php';

// false = write to file, true = dynamic
$dynamic = true;

// This only matters if you're writing to the file
$filename = 'sitemap.xml';


// Get the topics
$result = $db->query('SELECT t.id as topic_id, last_post, sticky, num_replies FROM '.$db->prefix.'topics AS t LEFT JOIN '.$db->prefix.'forum_perms AS fp ON (fp.forum_id=t.forum_id AND fp.group_id=3) WHERE (fp.read_forum IS NULL OR fp.read_forum=1) AND t.moved_to IS NULL ORDER BY last_post DESC') or error('Unable to fetch topic list', __FILE__, __LINE__, $db->error());

// Get the forums
$result2 = $db->query('SELECT f.id as forum_id, last_post, num_topics FROM '.$db->prefix.'forums AS f LEFT JOIN '.$db->prefix.'forum_perms AS fp ON (fp.forum_id=f.id AND fp.group_id=3) WHERE fp.read_forum IS NULL OR fp.read_forum=1 ORDER BY f.id DESC') or error('Unable to fetch forum list', __FILE__, __LINE__, $db->error());


$output = '<?xml version="1.0" encoding="UTF-8"?>'."\n";
$output .= '<urlset xmlns="http://www.google.com/schemas/sitemap/0.84">'."\n";

// The board itself
$output .= "<url>\n";
$output .= "\t<loc>".$pun_config['o_base_url']."/</loc>\n";
$output .= "\t<lastmod>".gmdate('Y-m-d\TH:i:s+00:00', time())."</lastmod>\n";
$output .= "\t<priority>1.0</priority>\n";
$output .= "</url>\n\n";


// Output the data for the forums
while ($cur_forum = $db->fetch_assoc($result2))
{
    $lastmodified = gmdate('Y-m-d\TH:i:s+00:00', $cur_forum['last_post']);
    $viewforum = 'viewforum.php?id='.$cur_forum['forum_id'];
    $priority = '1.0';

    if ($cur_forum['num_topics'] >= $pun_config['o_disp_topics_default'])
    {
        $num_pages = ceil($cur_forum['num_topics'] / $pun_config['o_disp_topics_default']);

        //Don't add &p=1 to the firs page
        $output .= "<url>\n";
        $output .= "\t<loc>".$pun_config['o_base_url']."/".$viewforum."</loc>\n";
        $output .= "\t<lastmod>$lastmodified</lastmod>\n";
        $output .= "\t<priority>$priority</priority>\n";
        $output .= "</url>\n\n";

        //Add page number for subsequent pages
        for ($i = 2; $i <= $num_pages; $i++)
        {
            $output .= "<url>\n";
            $output .= "\t<loc>".$pun_config['o_base_url']."/".$viewforum."&p=".$i."</loc>\n";
            $output .= "\t<lastmod>$lastmodified</lastmod>\n";
            $output .= "\t<priority>$priority</priority>\n";
            $output .= "</url>\n\n";
        }
    }
    else
    {
        $output .= "<url>\n";
        $output .= "\t<loc>".$pun_config['o_base_url']."/".$viewforum."</loc>\n";
        $output .= "\t<lastmod>$lastmodified</lastmod>\n";
        $output .= "\t<priority>$priority</priority>\n";
        $output .= "</url>\n\n";
    }
}

// Output the data for the topics
while ($cur_topic = $db->fetch_assoc($result))
{
    $lastmodified = gmdate('Y-m-d\TH:i:s+00:00', $cur_topic['last_post']);
    $viewtopic = 'viewtopic.php?id='.$cur_topic['topic_id'];
    $priority = ($cur_topic['sticky'] == '1') ? '1.0' : '0.5';

    if ($cur_topic['num_replies'] >= $pun_config['o_disp_posts_default'])
    {
        // We add one because the first post is not counted as a reply but needs to be
        // taken into account for display
        $num_pages = ceil(($cur_topic['num_replies'] + 1) / $pun_config['o_disp_posts_default']);

        $output .= "<url>\n";
        $output .= "\t<loc>".$pun_config['o_base_url']."/".$viewtopic."</loc>\n";
        $output .= "\t<lastmod>$lastmodified</lastmod>\n";
        $output .= "\t<priority>$priority</priority>\n";
        $output .= "</url>\n\n";

        for ($i = 2; $i <= $num_pages; $i++)
        {
            $output .= "<url>\n";
            $output .= "\t<loc>".$pun_config['o_base_url']."/".$viewtopic."&p=".$i."</loc>\n";
            $output .= "\t<lastmod>$lastmodified</lastmod>\n";
            $output .= "\t<priority>$priority</priority>\n";
            $output .= "</url>\n\n";
        }
    }
    else
    {
        $output .= "<url>\n";
        $output .= "\t<loc>".$pun_config['o_base_url']."/".$viewtopic."</loc>\n";
        $output .= "\t<lastmod>$lastmodified</lastmod>\n";
        $output .= "\t<priority>$priority</priority>\n";
        $output .= "</url>\n\n";
    }
}
$output .= "</urlset>\n";

// If we chose dynamic, we output the sitemap
// Otherwise, we write it to the file
if ($dynamic)
{
    header('Content-type: application/xml');
    echo $output;
}
else
{
    $file = fopen($filename, "w");
    fwrite($file, $output);
    fclose($file);
    echo "Done";
}
?>
http://www.info-mob.com/forum/ - Croatian forum only, don't bother if you don't speak Croatian :)

Re: Google Sitemaps

OK, uploaded a fixed version as 1.0.2b

22

Re: Google Sitemaps

Great smile

http://www.info-mob.com/forum/ - Croatian forum only, don't bother if you don't speak Croatian :)

23

Re: Google Sitemaps

I have a leading space before <xml version="1.0" encoding="UTF-8"?> in generated file.
why?
I still have an older versin of furum

24

Re: Google Sitemaps

Hm, don't know, I don't have one...

http://www.info-mob.com/forum/ - Croatian forum only, don't bother if you don't speak Croatian :)

25

Re: Google Sitemaps

hmm. very strange sad