Topic: Very big Handler_read_rnd_next when using PunBB ?

Hi,

A have checked Handler_read_rnd_next in phpmyadmin and I have very big value 90 M after 50 minut when mysql is restarted.

The number of requests to read the next row in the data file. This is high if you are doing a lot of table scans. Generally this suggests that your tables are not properly indexed or that your queries are not written to take advantage of the indexes you have.

What's is wrong ?

Darmowe forum - Polish free forum hosting

Re: Very big Handler_read_rnd_next when using PunBB ?

That's not nearly enough information to figure out the problem wink
For one thing, is PunBB the only thing running on the server? If not, then it could very well be something else on the server.
If PunBB is the only possible cause, then you'll need to track down which queries are doing full table scans. If you're using MySQL 5.0, you can log queries that don't use indexes: otherwise you'll have to find some other way (maybe just the normal slow query log with you running the queries through EXPLAIN to find the issue?)

Re: Very big Handler_read_rnd_next when using PunBB ?

I'm running on my server only PunBB forums. I'm using MySQL 5.0.30.

How should I find queries which are doing full table scans. How could I log it ?

Darmowe forum - Polish free forum hosting

Re: Very big Handler_read_rnd_next when using PunBB ?

http://dev.mysql.com/doc/refman/5.0/en/ … y-log.html

In MySQL 5.0, queries that do not use indexes are logged in the slow query log if the --log-queries-not-using-indexes option is specified. See Section 5.2.2, ?Command Options?.

Re: Very big Handler_read_rnd_next when using PunBB ?

So I have only 12 slow queries after 2 hours

Uptime:                 2 hours 25 sec

Threads: 1  Questions: 893621  Slow queries: 12  Opens: 408456  Flush tables: 1  Open tables: 2048  Queries per second avg: 123.685

But in /var/log/mysql/mysql-slow.log I have queries like that:

# Time: 070123 22:12:30
# User@Host: pun[pun] @ localhost []
# Query_time: 11  Lock_time: 0  Rows_sent: 17  Rows_examined: 1594191
use pun_klasa3h;
SELECT u.email, u.title, u.url, u.yahoo, u.location, u.use_avatar, u.aim, u.msn, u.signature, u.email_setting, u.num_posts, u.registered, u.admin_note, SUM(r.rep_plus) AS count_rep_plus, SUM(r.rep_minus) AS count_rep_minus, u.reputation_enable, p.id, p.poster AS username, p.poster_id, p.poster_ip, p.poster_email, p.message, p.hide_smilies, p.posted, p.edited, p.edited_by, g.g_id, g.g_user_title, o.user_id AS is_online FROM posts AS p INNER JOIN users AS u ON u.id=p.poster_id INNER JOIN groups AS g ON g.g_id=u.group_id LEFT JOIN online AS o ON (o.user_id=u.id AND o.user_id!=1 AND o.idle=0) LEFT JOIN reputation as r ON (r.user_id=u.id) WHERE p.topic_id=101 GROUP BY p.id ORDER BY p.id LIMIT 2000,25;
Darmowe forum - Polish free forum hosting

Re: Very big Handler_read_rnd_next when using PunBB ?

Well, there are a couple possible issues there: before I list them though, I'd like you to show the output of the following SQL statement:

explain SELECT u.email, u.title, u.url, u.yahoo, u.location, u.use_avatar, u.aim, u.msn, u.signature, u.email_setting, u.num_posts, u.registered, u.admin_note, SUM(r.rep_plus) AS count_rep_plus, SUM(r.rep_minus) AS count_rep_minus, u.reputation_enable, p.id, p.poster AS username, p.poster_id, p.poster_ip, p.poster_email, p.message, p.hide_smilies, p.posted, p.edited, p.edited_by, g.g_id, g.g_user_title, o.user_id AS is_online FROM posts AS p INNER JOIN users AS u ON u.id=p.poster_id INNER JOIN groups AS g ON g.g_id=u.group_id LEFT JOIN online AS o ON (o.user_id=u.id AND o.user_id!=1 AND o.idle=0) LEFT JOIN reputation as r ON (r.user_id=u.id) WHERE p.topic_id=101 GROUP BY p.id ORDER BY p.id LIMIT 2000,25;

Re: Very big Handler_read_rnd_next when using PunBB ?

Here you are

+----+-------------+-------+--------+------------------------------------+--------------------+---------+-------------------------+------+---------------------------------+
| id | select_type | table | type   | possible_keys                      | key                | key_len | ref                     | rows | Extra                           |
+----+-------------+-------+--------+------------------------------------+--------------------+---------+-------------------------+------+---------------------------------+
|  1 | SIMPLE      | o     | system | online_user_id_idx                 | NULL               | NULL    | NULL                    |    0 | const row not found             |
|  1 | SIMPLE      | p     | ref    | posts_topic_id_idx,posts_multi_idx | posts_topic_id_idx | 4       | const                   | 2000 | Using temporary; Using filesort |
|  1 | SIMPLE      | u     | eq_ref | PRIMARY                            | PRIMARY            | 4       | pun_klasa3h.p.poster_id |    1 |                                 |
|  1 | SIMPLE      | r     | ALL    | NULL                               | NULL               | NULL    | NULL                    |  716 |                                 |
|  1 | SIMPLE      | g     | eq_ref | PRIMARY                            | PRIMARY            | 4       | pun_klasa3h.u.group_id  |    1 |                                 |
+----+-------------+-------+--------+------------------------------------+--------------------+---------+-------------------------+------+---------------------------------+
5 rows in set (0.18 sec)
Darmowe forum - Polish free forum hosting

Re: Very big Handler_read_rnd_next when using PunBB ?

aha!
OK, so two more queries I need the output of:

describe reputation
show indexes from reputation

Re: Very big Handler_read_rnd_next when using PunBB ?

Here you are

mysql> describe reputation;
+--------------+---------------------+------+-----+---------+----------------+
| Field        | Type                | Null | Key | Default | Extra          |
+--------------+---------------------+------+-----+---------+----------------+
| id           | int(10) unsigned    | NO   | PRI | NULL    | auto_increment |
| user_id      | int(10) unsigned    | NO   |     | 0       |                |
| from_user_id | int(10) unsigned    | NO   |     | 0       |                |
| time         | int(10) unsigned    | NO   |     | 0       |                |
| post_id      | int(10) unsigned    | NO   | MUL | 0       |                |
| reason       | text                | NO   |     |         |                |
| rep_plus     | tinyint(1) unsigned | NO   |     | 0       |                |
| rep_minus    | tinyint(1) unsigned | NO   |     | 0       |                |
| topics_id    | int(10) unsigned    | NO   | MUL | 0       |                |
+--------------+---------------------+------+-----+---------+----------------+
9 rows in set (0.19 sec)
mysql> show indexes from reputation;
+------------+------------+-----------------------+--------------+--------------                                             +-----------+-------------+----------+--------+------+------------+---------+
| Table      | Non_unique | Key_name              | Seq_in_index | Column_name                                               | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+------------+------------+-----------------------+--------------+--------------                                             +-----------+-------------+----------+--------+------+------------+---------+
| reputation |          0 | PRIMARY               |            1 | id                                                        | A         |         717 |     NULL | NULL   |      | BTREE      |         |
| reputation |          1 | rep_post_id_idx       |            1 | post_id                                                   | A         |         717 |     NULL | NULL   |      | BTREE      |         |
| reputation |          1 | rep_multi_user_id_idx |            1 | topics_id                                                 | A         |          79 |     NULL | NULL   |      | BTREE      |         |
| reputation |          1 | rep_multi_user_id_idx |            2 | from_user_id                                              | A         |         179 |     NULL | NULL   |      | BTREE      |         |
+------------+------------+-----------------------+--------------+--------------                                             +-----------+-------------+----------+--------+------+------------+---------+
4 rows in set (0.11 sec)
Darmowe forum - Polish free forum hosting

Re: Very big Handler_read_rnd_next when using PunBB ?

OK, create an index like so

create index rep_user_id_idx on reputation(user_id)

11

Re: Very big Handler_read_rnd_next when using PunBB ?

Ok I will do it.

Darmowe forum - Polish free forum hosting

12

Re: Very big Handler_read_rnd_next when using PunBB ?

I think about next index in table online.

LEFT JOIN online AS o ON (o.user_id=u.id AND o.user_id!=1 AND o.idle=0)

Don't you think it should be index on idle in online ?

Darmowe forum - Polish free forum hosting

Re: Very big Handler_read_rnd_next when using PunBB ?

No, since there's an index on user_id. You don't need to index every column that's used in a query wink

14

Re: Very big Handler_read_rnd_next when using PunBB ?

Thanks for help smile My Handler_read_rnd_next after 4 hours is only 6 688 k big_smile

Darmowe forum - Polish free forum hosting

Re: Very big Handler_read_rnd_next when using PunBB ?

Nice smile

16

Re: Very big Handler_read_rnd_next when using PunBB ?

I have another slow queries with the same question:

# Query_time: 19  Lock_time: 0  Rows_sent: 15  Rows_examined: 158486
use pun_klasa3h;
SELECT u.email, u.title, u.url, u.yahoo, u.location, u.use_avatar, u.aim, u.msn, u.signature, u.email_setting, u.num_posts, u.registered, u.admin_note, SUM(r.rep_plus) AS count_rep_plus, SUM(r.rep_minus) AS count_rep_minus, u.reputation_enable, p.id, p.poster AS username, p.poster_id, p.poster_ip, p.poster_email, p.message, p.hide_smilies, p.posted, p.edited, p.edited_by, g.g_id, g.g_user_title, o.user_id AS is_online FROM posts AS p INNER JOIN users AS u ON u.id=p.poster_id INNER JOIN groups AS g ON g.g_id=u.group_id LEFT JOIN online AS o ON (o.user_id=u.id AND o.user_id!=1 AND o.idle=0) LEFT JOIN reputation as r ON (r.user_id=u.id) WHERE p.topic_id=101 GROUP BY p.id ORDER BY p.id LIMIT 2100,25;
mysql> explain SELECT u.email, u.title, u.url, u.yahoo, u.location, u.use_avatar, u.aim, u.msn, u.signature, u.email_setting, u.num_posts, u.registered, u.admin_note, SUM(r.rep_plus) AS count_rep_plus, SUM(r.rep_minus) AS count_rep_minus, u.reputation_enable, p.id, p.poster AS username, p.poster_id, p.poster_ip, p.poster_email, p.message, p.hide_smilies, p.posted, p.edited, p.edited_by, g.g_id, g.g_user_title, o.user_id AS is_online FROM posts AS p INNER JOIN users AS u ON u.id=p.poster_id INNER JOIN groups AS g ON g.g_id=u.group_id LEFT JOIN online AS o ON (o.user_id=u.id AND o.user_id!=1 AND o.idle=0) LEFT JOIN reputation as r ON (r.user_id=u.id) WHERE p.topic_id=101 GROUP BY p.id ORDER BY p.id LIMIT 2100,25;
+----+-------------+-------+------+------------------------------------+--------------------+---------+------------------------+------+---------------------------------+
| id | select_type | table | type | possible_keys                      | key                | key_len | ref                    | rows | Extra                           |
+----+-------------+-------+------+------------------------------------+--------------------+---------+------------------------+------+---------------------------------+
|  1 | SIMPLE      | u     | ALL  | PRIMARY                            | NULL               | NULL    | NULL                   |   26 | Using temporary; Using filesort |
|  1 | SIMPLE      | o     | ref  | online_user_id_idx                 | online_user_id_idx | 4       | pun_klasa3h.u.id       |    2 |                                 |
|  1 | SIMPLE      | g     | ALL  | PRIMARY                            | NULL               | NULL    | NULL                   |    3 | Using where                     |
|  1 | SIMPLE      | p     | ref  | posts_topic_id_idx,posts_multi_idx | posts_multi_idx    | 8       | pun_klasa3h.u.id,const |    6 |                                 |
|  1 | SIMPLE      | r     | ref  | rep_user_id_idx                    | rep_user_id_idx    | 4       | pun_klasa3h.u.id       |   43 |                                 |
+----+-------------+-------+------+------------------------------------+--------------------+---------+------------------------+------+---------------------------------+
5 rows in set (0.00 sec)
mysql> describe reputation;
+--------------+---------------------+------+-----+---------+----------------+
| Field        | Type                | Null | Key | Default | Extra          |
+--------------+---------------------+------+-----+---------+----------------+
| id           | int(10) unsigned    | NO   | PRI | NULL    | auto_increment |
| user_id      | int(10) unsigned    | NO   | MUL | 0       |                |
| from_user_id | int(10) unsigned    | NO   |     | 0       |                |
| time         | int(10) unsigned    | NO   |     | 0       |                |
| post_id      | int(10) unsigned    | NO   | MUL | 0       |                |
| reason       | text                | NO   |     |         |                |
| rep_plus     | tinyint(1) unsigned | NO   |     | 0       |                |
| rep_minus    | tinyint(1) unsigned | NO   |     | 0       |                |
| topics_id    | int(10) unsigned    | NO   | MUL | 0       |                |
+--------------+---------------------+------+-----+---------+----------------+
9 rows in set (0.01 sec)
mysql> show indexes from reputation;
+------------+------------+-----------------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+
| Table      | Non_unique | Key_name              | Seq_in_index | Column_name  | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+------------+------------+-----------------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+
| reputation |          0 | PRIMARY               |            1 | id           | A         |         735 |     NULL | NULL   |      | BTREE      |         |
| reputation |          1 | rep_post_id_idx       |            1 | post_id      | A         |         735 |     NULL | NULL   |      | BTREE      |         |
| reputation |          1 | rep_multi_user_id_idx |            1 | topics_id    | A         |          81 |     NULL | NULL   |      | BTREE      |         |
| reputation |          1 | rep_multi_user_id_idx |            2 | from_user_id | A         |         183 |     NULL | NULL   |      | BTREE      |         |
| reputation |          1 | rep_user_id_idx       |            1 | user_id      | A         |          17 |     NULL | NULL   |      | BTREE      |         |
+------------+------------+-----------------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+
5 rows in set (0.00 sec)
Darmowe forum - Polish free forum hosting

Re: Very big Handler_read_rnd_next when using PunBB ?

Well, reputation isn't the issue here
Have you made any other changes to the DB since I had you create that index?

18

Re: Very big Handler_read_rnd_next when using PunBB ?

No, only rep_user_id_idx on reputation(user_id)

Darmowe forum - Polish free forum hosting

Re: Very big Handler_read_rnd_next when using PunBB ?

OK, could you run

show indexes from posts
show indexes from users
show indexes from groups

20

Re: Very big Handler_read_rnd_next when using PunBB ?

mysql> show indexes from posts;
+-------+------------+--------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name           | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+-------+------------+--------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| posts |          0 | PRIMARY            |            1 | id          | A         |        4555 |     NULL | NULL   |      | BTREE      |         |
| posts |          1 | posts_topic_id_idx |            1 | topic_id    | A         |         198 |     NULL | NULL   |      | BTREE      |         |
| posts |          1 | posts_multi_idx    |            1 | poster_id   | A         |          18 |     NULL | NULL   |      | BTREE      |         |
| posts |          1 | posts_multi_idx    |            2 | topic_id    | A         |         759 |     NULL | NULL   |      | BTREE      |         |
+-------+------------+--------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
4 rows in set (0.11 sec)
mysql> show indexes from users;
+-------+------------+----------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name             | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+-------+------------+----------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| users |          0 | PRIMARY              |            1 | id          | A         |          26 |     NULL | NULL   |      | BTREE      |         |
| users |          1 | users_registered_idx |            1 | registered  | A         |          26 |     NULL | NULL   |      | BTREE      |         |
| users |          1 | users_username_idx   |            1 | username    | A         |          26 |        8 | NULL   |      | BTREE      |         |
+-------+------------+----------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
3 rows in set (0.01 sec)
mysql> show indexes from groups;
+--------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table  | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+--------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| groups |          0 | PRIMARY  |            1 | g_id        | A         |           4 |     NULL | NULL   |      | BTREE      |         |
+--------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
1 row in set (0.06 sec)
Darmowe forum - Polish free forum hosting

Re: Very big Handler_read_rnd_next when using PunBB ?

That's really odd: there is some really poor query optimizing being done there.
I think the issue might be the group by statement: could you try removing some of the reputation mod's edits to the query (so the group by, the join, and the reputation table's columns) and posting an EXPLAIN for that query?

22

Re: Very big Handler_read_rnd_next when using PunBB ?

This is query without reputation mod:

mysql> explain SELECT u.email, u.title, u.url, u.yahoo, u.location, u.use_avatar, u.aim, u.msn, u.signature, u.email_setting, u.num_posts, u.registered, u.admin_note, p.id, p.poster AS username, p.poster_id, p.poster_ip, p.poster_email, p.message, p.hide_smilies, p.posted, p.edited, p.edited_by, g.g_id, g.g_user_title, o.user_id AS is_online FROM posts AS p INNER JOIN users AS u ON u.id=p.poster_id INNER JOIN groups AS g ON g.g_id=u.group_id LEFT JOIN online AS o ON (o.user_id=u.id AND o.user_id!=1 AND o.idle=0) WHERE p.topic_id=101 GROUP BY p.id ORDER BY p.id LIMIT 2100,25;
+----+-------------+-------+--------+------------------------------------+-----------------+---------+------------------------+------+---------------------------------+
| id | select_type | table | type   | possible_keys                      | key             | key_len | ref                    | rows | Extra                           |
+----+-------------+-------+--------+------------------------------------+-----------------+---------+------------------------+------+---------------------------------+
|  1 | SIMPLE      | o     | system | online_user_id_idx                 | NULL            | NULL    | NULL                   |    0 | const row not found             |
|  1 | SIMPLE      | u     | ALL    | PRIMARY                            | NULL            | NULL    | NULL                   |   26 | Using temporary; Using filesort |
|  1 | SIMPLE      | g     | ALL    | PRIMARY                            | NULL            | NULL    | NULL                   |    3 | Using where                     |
|  1 | SIMPLE      | p     | ref    | posts_topic_id_idx,posts_multi_idx | posts_multi_idx | 8       | pun_klasa3h.u.id,const |    6 |                                 |
+----+-------------+-------+--------+------------------------------------+-----------------+---------+------------------------+------+---------------------------------+
4 rows in set (0.47 sec)
Darmowe forum - Polish free forum hosting

Re: Very big Handler_read_rnd_next when using PunBB ?

Talk to your host: I really have no idea why your query is being optimized that poorly (especially when it was working perfectly before)

Re: Very big Handler_read_rnd_next when using PunBB ?

On second thought, I have one idea: try running these

analyze table users
analyze table posts
analyze table groups

25

Re: Very big Handler_read_rnd_next when using PunBB ?

My host haven't idea too sad

mysql> analyze table users;
+-------------------+---------+----------+----------+
| Table             | Op      | Msg_type | Msg_text |
+-------------------+---------+----------+----------+
| pun_klasa3h.users | analyze | status   | OK       |
+-------------------+---------+----------+----------+
1 row in set (0.04 sec)
mysql> analyze table posts;
+-------------------+---------+----------+----------+
| Table             | Op      | Msg_type | Msg_text |
+-------------------+---------+----------+----------+
| pun_klasa3h.posts | analyze | status   | OK       |
+-------------------+---------+----------+----------+
1 row in set (0.74 sec)
mysql> analyze table groups;
+--------------------+---------+----------+-----------------------------+
| Table              | Op      | Msg_type | Msg_text                    |
+--------------------+---------+----------+-----------------------------+
| pun_klasa3h.groups | analyze | status   | Table is already up to date |
+--------------------+---------+----------+-----------------------------+
1 row in set (0.00 sec)
Darmowe forum - Polish free forum hosting