1 (edited by pedrotuga 2006-10-25 18:40)

Topic: newsbot - post RSS news

hey there...
I wanted to have rss news in my forums, one per topic so they can be commented and discussed. I think this is cool cause it allows you to keep a newsarchive on your forums.

ok... originaly i wrote the code to use myself with some settings mixed up inside the code, bad programing practive i know. Also the variable names, wich are very descriptive were writen in portuguese.

i went through the script and adapted it, the original version is working like a charm on my forums. I didnt test the modified version though so its possible that it has some kind of bug, i think not wink

If you need it just try it out, if you get some errors report them and i fix the problem.

You have to create a new table on your database and define some setings. The feed urls should be inserted in that table. The sql to create the table is in the code as a comment


OK, so this is how i put this working:
i created a folder in punbbroot and put this file there, you can name it whatever you feel like. I also put magpie and a modified copy of search_idx.php.
You can dload magpie here.
since this only works with mysql and i didnt had access to the database class, my modified search_idx.php has all the $db->function() replaced by the equivalent nativve mysql function. Like, $db->query() is replaced by mysql_query().

Ok... i know this is bad programing practice, i just wanted to have it working.
Anyway... any questions or trouble reply to this message.

<?php

//configuration=================================================================
//database===========================================
$db_host = '';
$db_name = '';
$db_username = '';
$db_password = '';

//other===============================================
$newstbot_name=''; //name of the user that make the news post
$newsbot_id='';  //id of that user
$newsforum_id=''; //the id of the forum where you wan the news to go
$follow_link_text='Read more at'; //change this if you want to display some other text
//==============================================================================

/*
sql for creating the extra table you need

create table feedsources(
id         int,
feed_url     varchar(255),
updated     int DEFAULT 0
);

*/


require('magpierss/rss_fetch.inc');
define(constantebot,1);
require "search_idx.php";


mysql_connect($db_host, $db_username, $db_password);
mysql_select_db($db_name);


$sql_get_feeds="select id, feed_url, updated from feedsources";
$resultado=mysql_query($sql_get_feeds);

$ii=0;
while ( $row=mysql_fetch_array($resultado) ){
    $feed_array[$ii]["feed"]=$row["feed_url"];
    $feed_array[$ii]["id"]=$row["id"];
    $feed_array[$ii]["updated"]=$row["updated"];
    $ii++;
}

$num_posts=0;

foreach ($feed_array as $feed_array_element){
    
    $feed_last_update        =    $feed_array_element["updated"];
    $url                =    $feed_array_element["feed"];
    $feed_id            =    $feed_array_element["id"];
    $rss                 =    fetch_rss($url);    

    $new_last_update = $feed_last_update;
    
    foreach ( $rss->items as $item ) {
    
                $timenow    = time();
                $date_string     = $item["pubdate"];
                $date_posted     = strtotime($date_string);
                
                if($date_posted > $new_last_update){
                    $new_last_update=$date_posted;
                }
                
                if($date_posted <= $feed_last_update){
                    
                    break;
                }
                
                $num_posts++;
                $topic_subject        =    $item['title'];
                $poster            =     $newstbot_name;
                $topic_forum_id        =    $newsforum_id;
                $poster_id        =    $newsbot_id;;

                $post_message        =     mysql_escape_string($item['description']);
                $post_message        =    $post_message.
                "<br /><br /><a href=\"".$item["link"]."\">$follow_link_text</a>";
                
    
                
                $sql="
                    insert into topics
                        (subject,
                        poster,
                        posted,
                        forum_id,
                        last_post_id,
                        last_poster)
                    values
                        ('$topic_subject',
                        '$poster',
                        $timenow,
                        $topic_forum_id,
                        $timenow,
                        '$poster')";
                        
                mysql_query($sql);
                
                $topic_id = mysql_insert_id();
                
                $sql="
                    insert into posts                
                        (poster,
                        poster_id,
                        message,
                        hide_smilies,
                        posted,
                        topic_id)
                    values
                        ('$poster',
                        $poster_id,'"
                        ."$post_message"
                        ."',
                        1,
                        $timenow,
                        $topic_id)";
                
                mysql_query($sql);
                $last_post_id = mysql_insert_id();
         
                $sql="update topics set last_post_id=$last_post_id, last_post=$date_posted where id=$topic_id";
                echo "$sql <br>";
                mysql_query($sql);
                
                $sql="
                    update forums
                    set
                        num_posts=num_posts+1,
                        num_topics=num_topics+1,
                        last_post=$timenow,
                        last_post_id=$last_post_id,
                        last_poster='$poster'
                        
                    where
                        id=$topic_forum_id";
                
                mysql_query($sql);

                                $post_message=strip_tags($post_message);
                update_search_index('post', $last_post_id, $post_message);
                               
                
                
        }
        
        $sql2="update feedsources set updated=$new_last_update where id=$feed_id";
        mysql_query($sql2);
        echo "$sql2 <br>";
        
    }
    
    $sql3="update users set num_posts=num_posts+$num_posts where id=$newsbot_id";
    mysql_query($sql3);
?>

2 (edited by Solovey 2006-10-25 05:40)

Re: newsbot - post RSS news

Thank you so much. I hope you fixed it after dinner. smile

Also, what is needed is a $db_prefix = 'pre_'; variable, as some people use database prifixes.

Regards.

Re: newsbot - post RSS news

I updated the opening topic.
Should work without problems, at least it does for me, though it's a quite tricky instaltion.

Any punbb guru could make this a module or something.
Caution:Some feeds have html, it will throw the source code on the output. Unless, of course, you bypass the parsing.

Re: newsbot - post RSS news

Can I see a demo somewhere ?
Thanks !

Re: newsbot - post RSS news

If you want people other than MySQL users to be able to use this, you should use PunBB's database class.

Looking for a certain modification for your forum? Please take a look here before posting.

Re: newsbot - post RSS news

grosbouff wrote:

Can I see a demo somewhere ?
Thanks !

Sorry, i am using it in a private board. It opens a new thread on the forum defined by you with the title being the feed item title and the body its description.

If you want people other than MySQL users to be able to use this, you should use PunBB's database class.

My interest is not having people using this. I am using it and i shared it. I did it my way and it works for the purpose i want.
As i am using punbb for free i think is good ethics to share my code as well even if is not following so many standarrds. The free software movement is based on giving and taking.

Re: newsbot - post RSS news

I've created a different version. It uses punbb's own methods for inserting a post and things so hopefully it'll work on multiple databases. You can see it in action on http://forums.kinkydress.com and feel free to use that forum for any comments/suggestions or rants smile Download URL: http://forums.kinkydress.com/downloads/ … ssnews.zip

8

Re: newsbot - post RSS news

harleyquine, interesting script, I can see some uses for it.

Is there a reason for using Magpie instead of SimplePie?

Re: newsbot - post RSS news

Simply because I've used Magpie before, no other reasons smile

Re: newsbot - post RSS news

Nice, you used the native db class!
I would just make two tiny changes: ou could use a foreach loop so the user doesnt have to define the number of feeds.

Re: newsbot - post RSS news

Thanks harleyquine, I'm using, perfect.

However just a small question.
I pulled it off another punbb forum and it shows like this sad

Forum: <a href="http://forums.elitistjerks.com/viewforum.php?id=8">Public Discussion</a><br />
Author: Kink<br />
Posted: Thu, 30 Nov 2006 04:37:23 -0800<br />
Last Post: Tue, 05 Dec 2006 07:23:45 -0800

Re: newsbot - post RSS news

Where exactly did you pull that feed from? Because thats the info of a whole feed rather than an item content.
Anyway, those html entities you see are there cause these bots do not parse html.

I did set a bot posting rss with html. Check out here how i did it:
http://www.punres.org/viewtopic.php?pid=9233#p9233

13

Re: newsbot - post RSS news

It seems to be unable to read UTF-8 characters ?

all that appear are "????"

14 (edited by pedrotuga 2006-12-11 15:01)

Re: newsbot - post RSS news

That might be tricky to solve.

First of all you have to keep in mind that this doesnt parse any HTML so all the html entities need to be parserd or will show somethinthing like

"caff&aacute";
instead of
"caffé"

Then you need to set your database charset to something compatible with your content.  For example, if your forums are in chinese a latin charset won't be apropriate.

Also you have to install those on your webserver if it doesnt have suport for it ( usualy this is not the problem as apache et al already come with every charset one can imagine, check it out anyway )

Easy to forget but important. Set the HTML document type charset to fit your needs. Wrong settings can mix your browser up.

Despite everything being working perfectly you might not be able to see those characters due to not having suport for a certain charset on your local machine. Make sure you operative system AND YOUR BROWSER have suport for that charset.

EDIT:
I don't know if you used my code or harleyquine's. If you used mine give a try to harleyquine's. It uses punbb's database class to make all database operations. I haven't look at punbb's  db class, if it does any parsing it might be prefered.

Re: newsbot - post RSS news

i took your code and modified it to suit my needs, great bit of code thanks for shairng it!

!

Re: newsbot - post RSS news

Nice to see that it has use beyond my needs wink

Hey... would it be ok if we used your instalation as a demo? I am using this on a private board that i cant post here.

Re: newsbot - post RSS news

if you mean use mine as a demo, sure, but keep in mind my version doesnt use an xml feed smile I made it so that when i post news it automaticaally posts a thread on the CORRECT board. so if its a contest, posts on my contests board, upcoming films, in my upcoming, etc. Got a few more bells I am going to add but your code gave me a great base.

Example, http://www.horror-movies.ca/Forum/viewtopic.php?id=1022

!

18

Re: newsbot - post RSS news

does someone got a working demo?????  and can u paste the code?
how does this create the seperate forum threads for the different catagories?

Q
example:
news
1.tech
2.worldly
3.local
4.govt

My stuff or my style might sux, but atleast I'm willing to help when I can.
Don't be stupid and help ! We are the stupid one's !!!

Re: newsbot - post RSS news

I have set up my script and fetch about 100 posts a day running it once through a crontab.
This script, as it is posts only in a category, but you can simply solve that.

Put an array into $newsforum_id rather than an intiger.
use a two dimension array in $feed_array being the first coordinate the forum_id of the feed.

wrap the code into one more foreach cicle to loop through the forums and replace $feed_array by $feed_array[$forum_id]

If all this sounds tricky or complicated i suggest you run multiple instances of ths script with different settings.

I could go into the code and change it a bit but i am currently out of time to test it.
If you go on and hack the code feel free to drop questions

Re: newsbot - post RSS news

harleyquine wrote:

I've created a different version. It uses punbb's own methods for inserting a post and things so hopefully it'll work on multiple databases. You can see it in action on http://forums.kinkydress.com and feel free to use that forum for any comments/suggestions or rants smile Download URL: http://forums.kinkydress.com/downloads/ … ssnews.zip

That download link doesn't work anymore to have a look on it.
Since this was last discussed, did anything new come up? Any clear demo and tutorial?
Thanks.

If you do what you've always done, you'll get what you've always gotten.

Re: newsbot - post RSS news

I had this on a forum that i was running, but it is no longer online... sorry.
Those interested will have to try it by themselves for the time being sad

Keep in mind that this is an extension for punbb 1.2.x.
It hasn't been tested in 1.3..

Re: newsbot - post RSS news

Maybe I'll post a request in the modifications forum, to see by any chance someone will be interested in making it.
Thanks pedrotuga smile.

If you do what you've always done, you'll get what you've always gotten.

Re: newsbot - post RSS news

Thank you so much for your work!

Re: newsbot - post RSS news

Solovey wrote:

Thank you so much for your work!

Did it work for you? Url?

If you do what you've always done, you'll get what you've always gotten.

Re: newsbot - post RSS news

Solovey, can you please re-eplain to me how it worked for you?
Can I add many RSS feeds? Can I determine a limit of feeds to be published a day?
Thanks in advance.

If you do what you've always done, you'll get what you've always gotten.