1 (edited by CodeXP 2005-06-15 17:36)

Topic: Cache output of extern.php, a little help?

Updated: Simplified cache check a little.

I was trying to get my own little PunBB based portal working (yes, I like re-inventing the wheel wink) and now I have a few questions about performance.

The thing is, I wanted to use extern.php on the page to display the latest 15 active topics. That part is all easy and good, but what I noticed was that this method isn't exactly the fastest way of doing things. There was a noticeable delay when refreshing the page, seeing as the script had to do a new SQL query.

I came to the conclusion that a system that kept the result in cache, for say 5min (put the expire time to low, and script would be useless), would be easier on the server. I came up with a pretty simple solution, but I want to ask if there are any even faster ways of doing this. Also, as I'm not the best PHP coder in the world (that's putting it mildly wink), I'm sure there are better ways if working with arrays and the if statements.

Here's what I got so far (don't worry about the static links etc., that's just temporarily until my admin plugin is done):

<?php
$extern_url = 'http://beta.tasarinan.com/'; // Full url to folder containing extern.php. Trailing slash required!
$s_cache = PUN_ROOT.'cache/sidebar.html'; // Sidebar cache file. Must be writable!
$age = '300'; // Max cache age in seconds (300sec = 5min)

// sidebar cache generator
    if (!file_exists($s_cache) || (time()-filemtime($s_cache) > $age)) {
        // Generate new cache because the old one doesn't exist, or is outdated!
        $tmp_php = file($extern_url.'extern.php?action=active') or die('Unable to fetch info from extern.php!');
        $fh = @fopen($s_cache, 'wb');
            foreach ($tmp_php as $tmp) {
                fwrite($fh, $tmp) or die('Could not write to cache file');    
            }
        fclose($fh);
    }
?>


<h2 id="sidebar_h2_top">Navigation</h2>
<div class="box">
    <div class="inbox">

    <ul>
    <li><a href="#">Link 1</a></li>
    <li><a href="#">Link 2</a></li>
    <li><a href="#">Link 3</a></li>
    <li><a href="#">Link 4</a></li>
    <li><a href="#">Link 5</a></li>
    </ul>

<h2>Active topics</h2>
    <ul>
<?php
    @(include($s_cache)) or die('Error fetching cache. Please check your configuration.');
?>
    </ul>
    </div>
</div>

If anyone want to test it, just change the variables at the top (script works on both linux and windows systems).

This way of doing it is a lot faster (on my server) than using just SQL, but as I said, I'm sure those of you more familiar with PHP can optimize it even further smile

Re: Cache output of extern.php, a little help?

Almost forgot, if you don't have defined PUN_ROOT, the script will fail (I have it wrapped in the forum layout). Just add the following at the top if you want to test the script:

define('PUN_ROOT','./');

Re: Cache output of extern.php, a little help?

I added a microtime counter on in the script to compare the real difference between them:

Cached results:
0.000535 (always somwhere around that number).

Non-cached results:
0.376675 (always somwhere around that number).

That was on my laptop.
Pentium 4 2.8Ghz
1GB Ram
Slow-ass 4200 rpm HDD

MySQL: 3.23.49
PHP: PHP/4.3.3
Apache: Apache/1.3.27 (Win32)

I'm not using the latest versions as you can see (this is just my laptop), but still.. That's a rather huge difference.

Here's the results from my webhost's server:

Cached results:
0.000423 (always somwhere around that number).

Non-cached results:
0.022419 (always somwhere around that number).

System:
(Unsure about speed of PC)

MySQL: 3.23.58
PHP: PHP/4.3.2
Apache: Apache 2.0

Re: Cache output of extern.php, a little help?

Try changing this and see if it helps the generation time a bit

5 (edited by CodeXP 2005-06-14 20:27)

Re: Cache output of extern.php, a little help?

It helps, but not enough I'm afraid.

Results:
0.019633

Of course, any query one can save helps, so I'm definetely going to continue using that tweak.. thanks smile

If you'd like to compare results for yourself, here's direct links to the sidebar (yes, it looks messed up separated from the rest of the page wink). Generation time are at the very bottom:
Cached version (try refresing the page if you get a lower result than expected. If cache is older than 300sec. it'll need to re-generate the cache)
Non-cached version

Re: Cache output of extern.php, a little help?

You should not be using MySQL 3 for any reason (in my opinion). MySQL 4+ is battle-tested and very very good. Plus, it already has a query cache.

Re: Cache output of extern.php, a little help?

Bwongar.com wrote:

You should not be using MySQL 3 for any reason (in my opinion). MySQL 4+ is battle-tested and very very good. Plus, it already has a query cache.

While true, that won't help much seeing as my host won't be upgrading anytime soon (blame it on the goddamn Ensim CP). In any case, the less SQL querys one has to run, no matter if using MySQL 3 or 4, the better smile

Re: Cache output of extern.php, a little help?

CodeXP: Well, it is a dynamic page tongue
However, let me check some other things...

Re: Cache output of extern.php, a little help?

Smartys wrote:

CodeXP: Well, it is a dynamic page tongue
However, let me check some other things...

Hehe, that's true smile

Still, I don't see any point in having the active topic list on my front page updating more than one each +/- 5min...

Re: Cache output of extern.php, a little help?

CodeXP wrote:

While true, that won't help much seeing as my host won't be upgrading anytime soon (blame it on the goddamn Ensim CP).

First of all, your host sucks tongue

Otherwise, for the most part, your code is fine. A little funky, sure, but it isn't necessarily bad.

CodeXP wrote:

In any case, the less SQL querys one has to run, no matter if using MySQL 3 or 4, the better smile

I think your insistence against MySQL is somewhat unwarranted. In many cases (talking MySQL 4+ here) a simple query is often faster than opening/reading/closing a file. This is especially enhanced with the aforementioned query cache. Many people find that the more queries they use, the slower their code is. I find that the more I rely on MySQL, the better and faster my code is.

Why is that? Because I know how to optimize my queries and MySQL Server. Take a look at this article I wrote if you want to learn more.

By the way, I'm not attacking you! I'm just trying to dispell the myth that MySQL is the culprit when most of the time it is bad code. (Or in your case, a much much too old version of MySQL tongue )

11

Re: Cache output of extern.php, a little help?

Good article Bwongar, that one goes to the "useful bookmarks" folder wink

That said, in my current situation, using a cache file will be the way to go. I *could* ask to get my site moved to one of their Cpanel servers, which is running much more up-to-date software, but that won't happen yet.
Also, in  a forum environment where there most likely will be new data added to the tables more frequently than once each 5min. (well, I'm hoping to get a more active community wink), I guess the query cache wouldn't help much anyway? Please correct me if I'm wrong though, as I'm always up for learning more smile

Re: Cache output of extern.php, a little help?

Actually, it does help quite a bit in a forum setting. If you look at the total number of views per topic in relation to how many posts/replies there are in said topic, you'll notice there is roughly a 1:12 (posts:views) ratio. So essentially just over 75% of the time (factoring in other... factors), the query cache is utilized. Plus, the cache will be utilized moreso in things that don't change so much, i.e. user info.

Does that help?

13

Re: Cache output of extern.php, a little help?

Bwongar.com wrote:

Actually, it does help quite a bit in a forum setting. If you look at the total number of views per topic in relation to how many posts/replies there are in said topic, you'll notice there is roughly a 1:12 (posts:views) ratio. So essentially just over 75% of the time (factoring in other... factors), the query cache is utilized. Plus, the cache will be utilized moreso in things that don't change so much, i.e. user info.

Does that help?

It sure does smile I think you misunderstood me a little though. I was just thinking of performance on the front page. Fetching active topics from all forums would most likely always return a new result set, so in that case, I wouldn't think you could improve performance by using query caches...?

Anyway, I upgraded my local MySQL installation to v4, but for some reason, even if I set the query cache size to 64M, it doesn't seem to go any faster with PunBB. PhpMyAdmin querys runs *much* faster though, so I know it's working, but the page generation times for PunBB seems to be the same. Any particular reason why I'm encountering this behaviour?

Re: Cache output of extern.php, a little help?

CodeXP wrote:

It sure does smile I think you misunderstood me a little though. I was just thinking of performance on the front page. Fetching active topics from all forums would most likely always return a new result set, so in that case, I wouldn't think you could improve performance by using query caches...?

I still believe it would. It would use the query to find all the newest posts, caching that query until a new post is made.

CodeXP wrote:

Anyway, I upgraded my local MySQL installation to v4, but for some reason, even if I set the query cache size to 64M, it doesn't seem to go any faster with PunBB. PhpMyAdmin querys runs *much* faster though, so I know it's working, but the page generation times for PunBB seems to be the same. Any particular reason why I'm encountering this behaviour?

I'm really not sure what could be causing this. It could be that since you have so few posts and topics on your laptop that it really doesn't make much difference period, considering the size of the database is VERY small. The overhead is probably not hit yet.

Secondly, it could be due to the fact that it is running on Windows. This is just a thought, as I have no experience with MySQL on a Windows box. MySQL was generally written for UNIX, ported to Windows as somewhat of an afterthought. Perhaps it doesn't read the configuration correctly?

Other than that, I don't know what to tell ya tongue

15

Re: Cache output of extern.php, a little help?

Bwongar.com wrote:

I'm really not sure what could be causing this. It could be that since you have so few posts and topics on your laptop that it really doesn't make much difference period, considering the size of the database is VERY small. The overhead is probably not hit yet.

Secondly, it could be due to the fact that it is running on Windows. This is just a thought, as I have no experience with MySQL on a Windows box. MySQL was generally written for UNIX, ported to Windows as somewhat of an afterthought. Perhaps it doesn't read the configuration correctly?

Other than that, I don't know what to tell ya tongue

True, that could be the problem. I tried doing a relatively small query in phpmyadmin by selecting around 200 posts, and I encountered the same thing. Caching works fast in phpmyadmin, but doesn't make any difference in PunBB. Tried the same with about 1000 posts, and again, the same result.

I'll install Apache, php + MySQL on my linux "box" and see...

Re: Cache output of extern.php, a little help?

My board on a first load takes about 0.15 seconds. After that one (hit refresh) and it takes 0.015 seconds. My personal server uses MySQL 4.0. The servers I use and maintain at work use MySQL extensively for content, tracking user data, searching, etc. and I can tell you this cache is a lifesaver!