I tried validating a forum page and got loads of errors which could be traced back to microsofts browser uploading windows-latin data as iso-8859-1 in a new forum message.
I fixed the problem, see the code below. Since my customized punbb code differs too much from the official tree, I can't give you a patchfile. The name 'demoronize' is inspired by Demoroniser. This bugfix only corrects the most common offending characters and replaces the other ones by question marks since I was too lazy to look up the unicode numbers of the other ones (the whole range 0x80 to 0xa0).
Extra function in include/functions.php (the . in "&"."#8364;" is because otherwise a € will show up in this post:
// remove non ISO-8859-1 characters from $text, return cleaned $text
function demoronize($text)
{
// common Microsoft non ISO-8859-1 characters
$search=array("#\x91#", "#\x92#", "#\x93#", "#\x94#", "#\x95#", "#\x96#", // smart quotes
"#\x80#" // euro
);
$replace=array("'", "'", "\"", "\"", "-", "-",
"&"."#8364;"
);
$text = preg_replace($search, $replace, $text);
// delete less common Windows-only characters
return preg_replace("#[\x80-\xa0]#", "?", $text);
}
In post.php, after the "validate bbcode syntax" block somewhere around line 170:
// and validate character set iso-8859-1
$message = demoronize($message);
In include/parser.php (the last line is added):
// Convert applicable characters to HTML entities
$text = pun_htmlspecialchars($text);
$text = demoronize($text); // remove non iso8859-1 characters that are left in the database