1 (edited by Frank H 2007-04-15 20:18)

Topic: Attachment Mod 2.0.1 by Frank H

##
##
##        Mod title:  Attachment Mod
##
##      Mod version:  2.0.1
##   Works on PunBB:  1.2.5
##     Release date:  2005-04-28
##           Author:  Frank Hagstrom (email removed, read my sig)
##
##      Description:  This mod will add the ability for attachments beeing
##                    posted in PunBB 1.2.5
##
##   Affected files:  delete.php
##                    edit.php
##                    moderate.php
##                    post.php
##                    viewtopic.php
##
##       Affects DB:  Yes
##
##            Notes:  This is the second Attachment Mod I have written. The
##                    earlier mod used the database to store the binary data,
##                    for a huge amount of files, this might start getting hard
##                    to make backups etc. of. (first mod was intended for few
##                    files, not gigabytes of data, but I have come closer and
##                    closer to this, and would like to be able to keep it on
##                    files instead.)
##                    
##                    To be able to get attachments, one need to enable upload
##                    of files in PHP, and set the max_file_size (and some
##                    other variables, so read documentation after installing
##                    the mod)
##                    There's now no longer any need of having large buffers
##                    for the database, so these can be returned to the values
##                    one had before installing the first mod (if you have that
##                    installed, but I guess you should do that after the files
##                    has been converted to disk files)
##                    
##                    There's also no need for editing php files to set options,
##                    these are set in the Administration interface, located in
##                    the Plugins menu. These are cached and should therefore be
##                    at least just as quick (perhaps quicker as they're
##                    combined with the forum config), so it's easier to
##                    administrate the mod now, adding icons, and such.
##                    
##                    Another great new thing done is that you only have to
##                    backup each file once, as there will never be two files
##                    with the same name in a directory. So one only need to
##                    download the new files from the subfolders, old deleted
##                    files will be emptied (0 bytes), but still be in the
##                    folders to keep new attachments to get the same name. As
##                    if they would, one would need to download all files during
##                    backup procedure.
##                    
##                    And as a further upgrade, posts are no longer limited to
##                    one attachment per post. I still have a limit of one file
##                    per opportunity (i.e. one on post creation, rest on edit),
##                    but the admin set the limit of max files per post, a per 
##                    group and per forum basis.
##                    
##                    I strongly suggest you read the whole documentation
##                    before start using the mod, the documentation is in the
##                    Administration interface. Or at the very least the first
##                    chapter!
##
##
##
##                    **** PREPARATIONS NEEDED TO BE DONE FIRST! ****
##
##                    1. Backup!
##                    2. Create a directory where you want the attachments to
##                       be stored. (Suggestion is somewhere the browser does
##                       NOT reach, if not bruteforcing to find files is
##                       possible)
##                    3. Make sure php is allowed to create files and
##                       directories in the above directory.
##                    
##                    Written by Frank H  
##                    on: 2005-04-12 17:11
##
##       DISCLAIMER:  Please note that "mods" are not officially supported by
##                    PunBB. Installation of this modification is done at your
##                    own risk. Backup your forum database and any and all
##                    applicable files before proceeding.
##
##

As always MAKE BACKUPS!!!

Backups should be simpler now, files only need to be downloaded once, as two attachments never has the same filename, even if an attachment is removed.

A couple of things to note:
1. The folder where the attachments are saved can (and should be) in a directory where browsers aren't allowed to be, but if that's not the case (for various reasons), an .htaccess and index.html is copied into each and every folder that the script makes for attachments. But beware, if the browser is allowed to see the index.html file, they may aswell bruteforce themselves to the attachments. So, my recommendation is to store the files in a place where the browsers aren't allowed.
2. If you have installed the "changeset 172" Rickard told about in the 1.2.5 release thread, then step 43 in readme is changed, and in step 44, just change the true to false after the query (and add the comment behind , so you have a mark that the attachment mod has altered that line) This affects PunBB 1.2.6+ aswell!!!
3. If you upgrade from pre 2.0 version, first remove all traces in the php files from the old mod (but keep database tables!), I suggest using a clean 1.2.5 install.
4. If bugs is surfaced, this post will be updated.
5. basefile.psd doesn't need to be uploaded, it's a Photoshop template if you want to create your own icons.
6. Those updating from 2.0 just read the update_2.0_to_2.0.1_readme.txt for instructions, it's a really simple upgrade, replace a few files, and run a small script smile
7. To test that upload works, use a tiny textfile (make a new textfile and write test in it, and save it). If that one works, but not other files, you probably haven't set the max upload size in "Alter Settings", read Documentation chapter 1.6 to figure out what value to use. (Using this small value to force people to set the settings according to their server, as 'every' server has it's own config)
8. "Blank page" can be the result of missing attach.php in all languages ... damac made me aware that it's a bit unclear that all languages need the attach.php file, so I'll make it a bit more clear in the next release...
9. Installation trouble can be caused by php running in safe mode, turning it off will make it work. (if you're getting an error on .htaccess beeing copied, but the chmod seems to be correct)

I've now made an Installation Guide! smile


Downloads:
Download from punres.org
Languages avaible for download: English(default), French, German, Greek, Norwegian, Russian, Simplified Chinese, Slovenian, Spanish & Swedish. Big thanks goes out to all the translators!
(when extracting lang files, never overwrite (they are all called the same in the zip), extract to different directories, or extract and move them one by one, all languages need their own attach.php files or the mod will give errors)



Critical Bugs:
SQLite: Installer doesn't work with SQLite (and might very well be that the whole mod in incompatible? (Anyone having time and knowledge of SQLite that would like to try to fix it?)
NULL byte vulnerability: The mod is probably vulernable to the NULL byte hack, that PunBB 1.2.13 was patched for, so the administrative plugin has been modified to solve that. Download the updated plugin from PunBB Resource, just overwrite your existing plugin. (can only be exploited by Administrators). More info in this post!

'Normal' Bugs:(can be related to different softwares etc. And not security related)
*new*(15th April 2007) Karinne noticed the readme says "index.htm" instead of "index.html". So please copy "index.html" to the attachment directory.
*new*(19th January 2007) Thesaint discovered an output bug in the administration interface for editing allowed files. Bugfix here.
*new*(24th November 2006) Koktsing had a problem where only 2'000'000 bytes were downloaded on his server, this was due to the fpassthru function probably hadn't enough memory to put the file, and just clipped at 2'000'000 bytes. One can solve it by applying a fix in attachment.php.
* (3rd April 2006) Fpouget reported that zip-files behaves odd in Internet Explorer if gzipped output (the zipfile has one file, that when unzipped and renamed to zip is the actual zipfile), works fine in Firefox though. Simple bugfix found in this post.
* Selad reported a typo in the readme.txt file "compressed.pnd to img/attach/" should be "compressed.png to img/attach/". Will be fixed in next release
* Gardell reported that attachment.php have a 'non translated' "Go back". Probably fixed in next release sometime ...




Fixed bugs(since release)
Fixed to 2.0.1 version:
* Max upload size is only checked agaisnt admins, so the rulecheck I had thought out, was only in my head and not in the code, typical roll Will make a fix. (but not critical, as the max upload size set in rules will still be the limiting factor, but if you have a small max upload, normal users can upload larger files (if allowed in ruleset), but not admins ...
* Performance issue when viewing attachment. Dexus reported having 5-7s extra waiting time for viewing attachments, but sending an extra header fixed it. See this post for info how to fix it. (happened on IE6 & Firefox 1.1 on an Apache 2 server). Will most probably be included in next release. This header is now in 2.0.1.

Fixed on the 2.0 release:
* readme in zip updated - (Step 32 in readme should say #---------[ 32. FIND (line: 358) ]--------------------------------------------   (thanks Dexus for noticing))
* readme in zip updated - (Step 15 in readme is wrong, use the step shown in this post (the large code block) (thanks Dexus for noticing))

Re: Attachment Mod 2.0.1 by Frank H

WOW! big_smile well done i'll be trying it later

Re: Attachment Mod 2.0.1 by Frank H

thank god.. haha

will try it later !

4 (edited by Dexus 2005-04-12 17:09)

Re: Attachment Mod 2.0.1 by Frank H

already tryed.. first of all - step 32 has wrong line (as in step 31)
the second. when admin - the error occur in edit.php in line about 220 ... when trying to edit existed message with attach
$attach_size=$pun_config[''];

Re: Attachment Mod 2.0.1 by Frank H

thanks, should be line 358 on step 32   (gong to update zip/7z with it)

hmm... going to check edit.php now

Re: Attachment Mod 2.0.1 by Frank H

hmm... at line 221 I have "$attach_allow_size=$pun_config['attach_max_size'];"  ... but you need to have run install_mod.php to be able to have those values cached (step 50)

check if your config cache has attach_max_size

7 (edited by Frank H 2005-04-12 17:35)

Re: Attachment Mod 2.0.1 by Frank H

oops, my bad! ... that line is wrong in the readme.txt, for some odd reason attach_max_size is missing ...

line 221 in edit.php should say

$attach_allow_size=$pun_config['attach_max_size'];

instead of

$attach_size=$pun_config[''];

Edit: sigh ... I'm going to recheck the readme.txt again, from top to bottom ... (I'll zip up the files modified aswell, so that it's possible to check oneself if something borks out sad)

use this as step 15 ...

#
#---------[ 15. AFTER, ADD ]--------------------------------------------------
#

//Attachment Mod 2.0 Block Start
//ok, first check the rules, so we know if the user may may upload more or delete potentially existing attachments
$attach_allow_delete=false;
$attach_allow_owner_delete=false;
$attach_allow_upload=false;
$attach_allowed=false;
$attach_allow_size=0;
$attach_per_post=0;
if($pun_user['g_id']==PUN_ADMIN){
    $attach_allow_delete=true;
    $attach_allow_owner_delete=true;
    $attach_allow_upload=true;
    $attach_allow_size=$pun_config['attach_max_size'];
    $attach_per_post=-1;
}else{
    $result_attach=$db->query('SELECT ar.rules,ar.size,ar.per_post,COUNT(f.id) FROM '.$db->prefix.'attach_2_rules AS ar, '.$db->prefix.'attach_2_files AS f, '.$db->prefix.'posts AS p, '.$db->prefix.'topics AS t WHERE group_id=\''.$pun_user['g_id'].'\' AND p.id = \''.$id.'\' AND t.id = p.topic_id AND ar.forum_id = t.forum_id GROUP BY f.post_id LIMIT 1')or error('Unable to fetch attachment rules and current number of attachments in post (#2)',__FILE__,__LINE__,$db->error());    
    if($db->num_rows($result_attach)==1){
        list($attach_rules,$attach_allow_size,$attach_per_post,$attach_num_attachments)=$db->fetch_row($result_attach);
        //may the user delete others attachments?
        $attach_allow_delete = attach_rules($attach_rules,ATTACH_DELETE);
        //may the user delete his/her own attachments?
        $attach_allow_owner_delete = attach_rules($attach_rules,ATTACH_OWNER_DELETE);
        //may the user upload new files?
        $attach_allow_upload = attach_rules($attach_rules,ATTACH_UPLOAD);
    }else{
        //no rules set, so nothing allowed
    }
}
$attach_output = '';
$attach_output_two = '';
//check if this post has attachments, if so make the appropiate output
if($attach_allow_delete||$attach_allow_owner_delete||$attach_allow_upload){
    $attach_allowed=true;
    $result_attach=$db->query('SELECT af.id, af.owner, af.filename, af.extension, af.size, af.downloads FROM '.$db->prefix.'attach_2_files AS af WHERE post_id=\''.$id.'\'')or error('Unable to fetch current attachments',__FILE__,__LINE__,$db->error());
    if($db->num_rows($result_attach)>0){
        //time for some output ... create the existing files ... 
        $i=0;
        while(list($attach_id,$attach_owner,$attach_filename,$attach_extension,$attach_size,$attach_downloads)=$db->fetch_row($result_attach)){
            if(($attach_owner==$pun_user['id']&&$attach_allow_owner_delete)||$attach_allow_delete)
                $attach_output .= '<br />'."\n".'<input type="checkbox" name="attach_delete_'.$i.'" value="'.$attach_id.'" />'.$lang_attach['Delete?'].' '.attach_icon($attach_extension).' <a href="./attachment.php?item='.$attach_id.'">'.$attach_filename.'</a>, '.$lang_attach['Size:'].' '.number_format($attach_size).' '.$lang_attach['bytes'].', '.$lang_attach['Downloads:'].' '.number_format($attach_downloads);
            else
                $attach_output_two .= '<br />'."\n".attach_icon($attach_extension).' <a href="./attachment.php?item='.$attach_id.'">'.$attach_filename.'</a>, '.$lang_attach['Size:'].' '.number_format($attach_size).' '.$lang_attach['bytes'].', '.$lang_attach['Downloads:'].' '.number_format($attach_downloads);
            $i++;
        }
        if(strlen($attach_output)>0)
            $attach_output = '<input type="hidden" name="attach_num_attachments" value="'.$db->num_rows($result_attach).'" />'.$lang_attach['Existing'] . $attach_output;
        if(strlen($attach_output_two)>0)
            $attach_output .= "<br />\n".$lang_attach['Existing2'] . $attach_output_two;
        $attach_output .= "<br />\n";
    }else{
        // we have not existing files
    }
}
//fix the 'new upload' field...
if($attach_allow_upload){
    if(strlen($attach_output)>0)$attach_output .= "<br />\n";
    if($attach_per_post==-1)$attach_per_post = '<em>unlimited</em>';
    $attach_output .= str_replace('%%ATTACHMENTS%%',$attach_per_post,$lang_attach['Upload'])."<br />\n".'<input type="hidden" name="MAX_FILE_SIZE" value="'.$attach_allow_size.'" /><input type="file" name="attached_file" size="80" />';
    
    
    
}
//Attachment Mod 2.0 Block End

Re: Attachment Mod 2.0.1 by Frank H

wooff... this mod at first looks complicated, but I've success on trying it.
It takes all of my energy while apply it and understands the doc (the biggest I've seen on mod) big_smile

Good works Frank.. smile

9 (edited by Dexus 2005-04-12 18:04)

Re: Attachment Mod 2.0.1 by Frank H

okay, Frank. I have no bugs left noticed.
But! smile Could you somehow make "attachment.php" a little bit more informative, when previewing images...
Simply to add file size and number of downloads, as in all other attachment lists?...

And ideas for future updates/releases:
- change format in administration of attachments, when listing all of them - to make it tabled (like topics).. also "pages" will be great.. smile IMHO it will be more user(admin)-friendly interface.
- also it will be good to have a tag for uploaded picture (by id).. a sort of [imgid=3] in a message body
- and these limitation of uplods (clearing) when previewing.. it isn't so good you know.. if attachment is big enough - re-uploading may take time...
- when creating a post - to "upload" file without actual posting, and posting after attaching all the file you need. Maybe link "my attachments" with a sort of tiny manager of user's attachments and posts with these attachments... when you could "reupload" some of them (newer version) or delete...  well.. a kinda... manager smile

P.S. Awesome mod!  smile

Re: Attachment Mod 2.0.1 by Frank H

the readme.txt is updated in the zips now ... I only think it was step 15 that had old code (I remember fixing that bug, but I must have missed to update the readme when I did), but I copy/pasted most stuff to be placed in the files ...

Re: Attachment Mod 2.0.1 by Frank H

Dexus wrote:

okay, Frank, could you somehow make "attachment.php" a little bit more informative, when previewing images...
Simply to add file size and number of downloads, as in all other lists...

and for the future - change format in administration of attachments, when listing all of them - to make it tabled (like topics).. also "pages" will be great..
but it for future.. smile IMHO it will be more user(admin)-friendly interface.

P.S. Awesome mod!  smile

Thanks smile

Well, I tried to make the mod 'mod-able' ... and considering how much time I've had recently, I might welcome if someone else asks to update the mod in the future ... so I tried to make the whole mod easy enough to get into ... and the whole documentation, well if someone wants to do a gallery, I suppose a bunch of code/functions can be used ... that's why I put the effort in to comment those things aswell...

About the extra stuff for imageviewer ... well I might, should be possible to combine into that same query already there, but first I'll prioritize bugs big_smile

I'm also going to test 'resume' on files later... but didn't add it to this release, as I haven't figured out a good way to test it yet ... but that might very well be included then aswell ... as it's the same file wink


At the admin interface, well ... we will see, not impossible wink

12 (edited by Dexus 2005-04-12 18:10)

Re: Attachment Mod 2.0.1 by Frank H

well, i've searched on the net about resuming broken uploads, and i'd found an idea, that it's near impossible, because clients(browsers) are unable to upload not from start...

Re: Attachment Mod 2.0.1 by Frank H

Dexus wrote:

- and these limitation of uplods (clearing) when previewing.. it isn't so good you know.. if attachment is big enough - re-uploading may take time...

- when creating a post - to "upload" file without actual posting, and posting after attaching all the file you need. Maybe link "my attachments" with a sort of tiny manager of user's attachments and posts with these attachments... when you could "reupload" some of them (newer version) or delete...  well.. a kinda... manager

I decided not to use any kind of cache/temporary stuff for uploads, as it will complicate things alot ... so the current functionality is kind of a result of that ... (this mod is IMHO not PunBB enough at some areas, too much fluff wink)

- also it will be good to have a tag for uploaded picture (by id).. a sort of [imgid=3] in a message body

extra code needed, and I have tried not to mess with stuff not directly connected to the actual attachments ... but it's quite easy to just write this in the post

[img=attachment.php?item=12&download=1]

, but one have to check what item each attachment got ...

Re: Attachment Mod 2.0.1 by Frank H

Dexus wrote:

well, i've searched on the net about resuming broken uploads, and i'd found an idea, that it's near impossible, because clients(browsers) are unable to upload not from start...

I know how to do, just that I haven't figured out a good way to debug it, I know what headers that needs to be sent, and what stuff the browser sends to the server, and I know how to get the correct stuff from the file ... but as I havent a way of doing proper debugging, it'll be a guessing game if it works, and that's why I don't have it now wink

Re: Attachment Mod 2.0.1 by Frank H

Will this work in 1.2.4?

16 (edited by Frank H 2005-04-13 01:00)

Re: Attachment Mod 2.0.1 by Frank H

probably, but 1.2.5 is security updates, so you should probably make the few updates to get a full 1.2.5 version (think it was only a few changes that affected the mod, and the hdiff really makes the job simple) ... it didn't take long to go from 1.2.4 to 1.2.5 for me ...
but some steps might be a little different, like the step 43/44, so you might need to look extra careful what all the replaces in the readme does ... but it's probably faster and better to upgrade to 1.2.5 before trying this mod.

Backups are especially important for this

17

Re: Attachment Mod 2.0.1 by Frank H

Thanks for the mod!

May I ask for a unified diff (diff -urN) against 1.2.5?

That note to the right of the zip-file with the modified files, does it say that those files are the result of applying this mod on against a clean 1.2.5? If so, I suppose I could generate the diff myself..

Re: Attachment Mod 2.0.1 by Frank H

Ulph wrote:

That note to the right of the zip-file with the modified files, does it say that those files are the result of applying this mod on against a clean 1.2.5? If so, I suppose I could generate the diff myself..

no, it says it is not a clean 1.2.5 with mod added, it acutally has the two changesets that came after 1.2.5 that Rickard told about in the 1.2.5 thread, but I think that's the only changes I've made (I had some other problem which I solved for my forum/server ... that might be in there aswell, don't remember what or where I changed stuff for that), so I suppose you can use those to make a diff file (I have written comments with attachment mod on all places where I have altered code, so if all the stuff the diff shows has comments about attachment mod, then I suppose it'll work)

(but do make backups wink)

19 (edited by Dexus 2005-04-13 07:48)

Re: Attachment Mod 2.0.1 by Frank H

okay, i've finished re-arranging list of attachments in administration.. also added forum/topic and owner.
also added info in Image View and dimentions info (width x height) of images.

Frank, can you tell me why image opening as embedded (through <img> tag) is so slow?
I mean some time pass before image actually occurs in "Image View"...

By the way, there is intresting detail: when you're guest - you're able to attach, and according to options, more then 1 attachment is possible, but due to "edit" nature of extra attaches, there is no possibility to attach more then one file to message for guests

Re: Attachment Mod 2.0.1 by Frank H

Dexus wrote:

Frank, can you tell me why image opening as embedded (through <img> tag) is so slow?
I mean some time pass before image actually occurs in "Image View"...

no idea why that would be the case, perhaps your server is under heavy load? I don't have any problems at my server... but it's slower than ordinary files, due to the security checks, and getting fileinfo, before sending stuff to the browser ...

By the way, there is intresting detail: when you're guest - you're able to attach, and according to options, more then 1 attachment is possible, but due to "edit" nature of extra attaches, there is no possibility to attach more then one file to message for guests

Yes, but that's because I do not want to add alot of extra code just for some temporary storage, as I've said a few times by now ... be happy that you now can attach more than one if you're registered wink ... last mod only had 1 per post, but many people wanted more, and this was the way to add the least amount of code to add that ability wink

I don't want to add every imaginable feature to this mod, if so I would have done so before releasing it, PunBB is a "lightweight" forum, and my mod is not that lightweight on some places, but adding temporary files etc. is defenetly 'fluff' to me

21

Re: Attachment Mod 2.0.1 by Frank H

no i have test base on local machine, it isn't loaded at all smile
this checks.. it takes about 5-7 seconds of odd waiting... after loading of page..

and about this "optimize" feature - as i see, it doesn't remove any files, that aren't in database, but in attachment folder... i really need this feature smile

And re-arranged attachment in administration looks like:
http://www.intramail.ru/~dex/myattadm.png

Re: Attachment Mod 2.0.1 by Frank H

5-7 seconds, that's weird, doesn't have anything like that here ... probably some configuration stuff? ...

optimize only affect database, to remove unused empty space in tables ... you can do the same in phpmyadmin or such ... i.e. it optimizes the tables
do read chapter 1.2 in the documentation ... the files deleted become 0 bytes, but left in the attachment folder to ease backups ...

23 (edited by Dexus 2005-04-13 13:38)

Re: Attachment Mod 2.0.1 by Frank H

about 0 lenght i know..
but for some reason there will be situation, when attaches in base - 30, but actually in attach folder there are 100 of them...
you don't see if file is used or not... and file aren't 0 size of length.. when you'd backed up base in one day, and files some later, when you're restoring base with files, there will be alot of files in folders, that won't be having appropriate row for it in base...
i mean tool that will zero all unused files in upload folders, but keeps their names...

yep 5-7 seconds.. and i don't know why....

Re: Attachment Mod 2.0.1 by Frank H

well, I might extend the orphan search for that, but it's a bit tricky if you have several subfolders ... so some thinking needed to make a smart way ... (but you shouldn't backup files and database at different occations, as written in documentation)

Database tables and files should be made backups of at about the same time (one before the other), and preferabely in Maintenece mode, so the files and database match up.

Re: Attachment Mod 2.0.1 by Frank H

Hrmm attachment mod seemed to go ok.. I have set rules and everything but when uploading a file it says redirecting.. then comes back to the post and the file is not there, attach plugin also shows no files..

any thoughts??