1 (edited by eric235u 2007-08-13 15:20)

Topic: php: updating database at end of script?

hi all.  i have a bittorrent scrape object on one page.  i then, require PUN_ROOT.'punscrape.php';, at the top of my index page.  so when a visitor goes to my index page the bittorrent tracker is scraped via libcurl.  the problem occurs when a tracker times out or is just slow to respond.  this problem is even worse when i have several torrents on several trackers that are slow to respond to my scrape request.  so...

how the heck to i run my scrape object after the users visits index?  this is a manged website and i don't have access to cron.  thanks for any tips!

just for amusement, here's my scrape page code.  it's a little noobish, but it functions.

<?php

/*

  This file isn't part of PunBB.  This file is part of 'puntracker'
  a bittorrent indexer modification for PunBB.

  PunBB is free software; you can redistribute it and/or modify it
  under the terms of the GNU General Public License as published
  by the Free Software Foundation; either version 2 of the License,
  or (at your option) any later version.

  PunBB is distributed in the hope that it will be useful, but
  WITHOUT ANY WARRANTY; without even the implied warranty of
  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
  GNU General Public License for more details.

  You should have received a copy of the GNU General Public License
  along with this program; if not, write to the Free Software
  Foundation, Inc., 59 Temple Place, Suite 330, Boston,
  MA  02111-1307  USA

*/

// let's keep tracker within punbb
define('PUN_ROOT', './');
require_once PUN_ROOT.'include/common.php';

// comment out after debugging
// error_reporting(E_ALL);

// the scrape object
class punscrape
{
    function do_scrape()
    {

    global $db;
    
/* for debugging purposes.  comment out.
    global $complete;
    global $downloads;
    global $incomplete;
    global $matches_complete;
    global $matches_downloaded;
    global $matches_incomplete;
    global $scrape_result;
    global $scrape_url;
    global $result1;
    global $cerror;
*/
    
// let's check if the torrent(s) need(s) to be scraped. (1.5 hours)
$table = $db->prefix.'puntracker';
$result1 = $db->query("SELECT announce FROM $table 
WHERE $table.last_scrape <= NOW() - INTERVAL 1.5 HOUR");

while ($row = $db->fetch_assoc($result1)) {

// if query returns a row, scrape url
    if (isset($row['announce'])) {

$scrape_url=$row['announce'];

// create a new cURL resource
$ch = curl_init();

// set URL and other appropriate options
curl_setopt($ch, CURLOPT_URL, $scrape_url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);  // to return text
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 6);  // at higher values i get "PHP Fatal error:  Allowed memory size of 20971520 bytes exhausted".  you may want to adjust these values for your own needs.
curl_setopt($ch, CURLOPT_TIMEOUT, 10);

// grab URL text and set $scrape_result
$scrape_result = curl_exec($ch);

// for debugging. comment out.
$cerror = curl_error ( $ch );

// close cURL resource, and free up system resources
curl_close($ch);

switch (isset($scrape_result)) {
case TRUE:

// let's extract complete (seeds), downloaded and incomplete (peers).
preg_match("<completei\d*>", $scrape_result, $matches_complete);
preg_match("<downloadedi\d*>", $scrape_result, $matches_downloaded);
preg_match("<incompletei\d*>", $scrape_result, $matches_incomplete);

$matches_complete = implode($matches_complete);
$matches_downloaded = implode($matches_downloaded);
$matches_incomplete = implode($matches_incomplete);

$complete = ltrim($matches_complete, "completei");
$downloads = ltrim($matches_downloaded, "downloadedi");
$incomplete = ltrim($matches_incomplete, "incompletei");

// if scrape returns text update mysql
if (is_numeric($complete)) {

$complete=mysql_real_escape_string($complete);
$downloads=mysql_real_escape_string($downloads);
$incomplete=mysql_real_escape_string($incomplete);

//  update mysql.
$table = $db->prefix.'puntracker';
$query = "UPDATE $table";
$query .= " SET complete = $complete, downloads = $downloads, incomplete = $incomplete, last_scrape = NOW()";
$query .= " WHERE announce = '$scrape_url'";
$result = mysql_query($query) or die('Query failed: ' . mysql_error());

}
    break;
    
case FALSE:
    break;
}

    }}

}}

// let's run punscrape.php
$updatedb = new punscrape;
$updatedb->do_scrape();

?>

Re: php: updating database at end of script?

In all honesty, if you don't have cron, I would make the script display a 1x1 transparent image and call the script in an image tag at the bottom of the page.
Alternately, if you can use the exec function, you can fork off that PHP script to run in the background.

3 (edited by eric235u 2007-08-14 13:52)

Re: php: updating database at end of script?

exec, that's a neat idea.  i'll see if my host has that function.

p.s. - what the f* is that?  // the link that was here has been removed.

Re: php: updating database at end of script?

Someone's idea of a bad joke. For the future, you should really put some kind of warnings about that link.

Re: php: updating database at end of script?

my mistake.  i assumed you did it on purpose and was just fooling around.

thanks again for the php tip.  i'm going to work on it today.

Re: php: updating database at end of script?

ah crap.  no 'Program Execution Functions' are supported.  too bad.  your suggestion of exec would have worked perfectly.

I would make the script display a 1x1 transparent image and call the script in an image tag at the bottom of the page.

mmmnnn....  i've never done something like that before.  i'll do a little reading.  thanks.

Re: php: updating database at end of script?

that works great!  cool.  thanks man.

example:  http://newmag.org/forum/puntracker.php