1 Topic by Vanslyde (edited by Vanslyde 2009-10-30 13:36)

  • From: Montreal
  • Registered: 2006-10-02
  • Posts: 144

Topic: Some posts being ignored when rebuilding the search index?

edit: please move to troubleshooting if needs be

Hi guys,

Quite recently I found out that some posts were being ignored when rebuilding my search index.

i.e. I can search and find a specific word in say, a particular topic of a particular section, but in some other cases the word I'm looking for simply just won't show.
I tested with plenty of words wihtout any ' or - or . next to them and the results seems to be clear; some of the posts on my forum do not get indexed and if they do, results simply won't show.

I'm under 1.2.22 and this problem is recent, so it may have occured when I moved from 1.2.16 to 1.2.22

Any questions, advice?

Thanks!

Vanslyde's Website

  • Slavok
  • PunBB Developer
  • Offline
  • From: Russia
  • Registered: 2008-06-11
  • Posts: 1,238

Re: Some posts being ignored when rebuilding the search index?

Are you sure that indexes of all topics were rebuild? Did any errors or notices appear during the process?

Slavok's Website

  • From: Montreal
  • Registered: 2006-10-02
  • Posts: 144

Re: Some posts being ignored when rebuilding the search index?

Slavok wrote:

Are you sure that indexes of all topics were rebuild?

No, because I have too many topics (it would be too long to test them all)
But I'm confident that they are. On several topics I can search and locate certain words in the first 10 posts but the subsequent ones (posts of that topic) aren't indexed (or won't show)

Slavok wrote:

Did any errors or notices appear during the process?

No, I emptied and rebuild the search index and I never got an error.
However, I saw some extra large topics get listed in the rebuild process for like half a second, so I doubt the whole topic gets through the complete process in those cases.

Vanslyde's Website

  • Registered: 2009-10-14
  • Posts: 4

Re: Some posts being ignored when rebuilding the search index?

Is it working now though?

  • From: Montreal
  • Registered: 2006-10-02
  • Posts: 144

Re: Some posts being ignored when rebuilding the search index?

davidbatchelor wrote:

Is it working now though?

No it's not, I emptied and rebuilded 3 times and that's why I created this thread.

Vanslyde's Website

  • From: Montreal
  • Registered: 2006-10-02
  • Posts: 144

Re: Some posts being ignored when rebuilding the search index?

Vanslyde wrote:
davidbatchelor wrote:

Is it working now though?

No it's not, I emptied and rebuilded 3 times and that's why I created this thread.

Just to clarify.
The indexing process is "working" all right.
The problem is it does not index everything.
My forum is in French (if that could be of any help/clue)

I'd like to get this fixed. I can't figure out why some topics won't get through the indexing process.

When I upgraded to 1.2.22 all folders got overwritten, maybe I forgot a particular setting I should have reapplied to one of them? (ie like the /img/avatars/ one)

If you know of any articles, topics, leads, shoot them up!

Vanslyde's Website

  • Slavok
  • PunBB Developer
  • Offline
  • From: Russia
  • Registered: 2008-06-11
  • Posts: 1,238

Re: Some posts being ignored when rebuilding the search index?

What is the encoding of your DB?
Did you try to manually find the problem word in the "search_words" table and the link to this word in the "search_matches" table? What is the word?

Slavok's Website

  • From: Montreal
  • Registered: 2006-10-02
  • Posts: 144

Re: Some posts being ignored when rebuilding the search index?

Hi, thanks for trying to help/direct me in the good direction again for a resolution

First, let me say that I'm not DB friendly/savvy. The less I need to go in the DB the better I'm off.  hmm 
So I don't think I changed any settings.

What is the encoding of your DB?
1-As per the first pic below, I think it's UTF-8 unicode, although the 2nd pick says it's latin-swedish.. (so I'm not too sure what to tell you)

Did you try to manually find the problem word in the "search_words" table and the link to this word in the "search_matches" table? What is the word?

3-As per the third pic below, I'm having a hard time finding how I could search a specific word. I can empty the tables, edit them, but I can't figure out how I could search IN them.
http://i119.photobucket.com/albums/o144/indiequebec/searchneedhelp.gif

The word I'm looking for (which is not indexed) is "Belinda" located here.
http://www.indiequebec.com/forums/viewt … d=396#p396
A sensible amount of posts below that one ^^ didn't get indexed... it also happened in some other threads and I'm wondering why.
Should you need more pics, infos, please let me know.  cool

Vanslyde's Website

  • Slavok
  • PunBB Developer
  • Offline
  • From: Russia
  • Registered: 2008-06-11
  • Posts: 1,238

Re: Some posts being ignored when rebuilding the search index?

Slavok wrote:

Did you try to manually find the problem word in the "search_words" table and the link to this word in the "search_matches" table?

To do this

  • Select the "search_words" table

  • Go to the tab "SQL"

  • Post this query in the field :

SELECT id
FROM search_words
WHERE word = 'Belinda'

  • Remember word id

  • Select the "search_matches" table

  • Go to the tab "SQL"

  • Post this query in the field :

SELECT *
FROM search_matches
WHERE word_id = <word_id>

Post here results of these queries.

Slavok's Website

10 Reply by Vanslyde (edited by Vanslyde 2009-10-30 13:35)

  • From: Montreal
  • Registered: 2006-10-02
  • Posts: 144

Re: Some posts being ignored when rebuilding the search index?

Thanks!

Slavok wrote:

  • Select the "search_words" table

  • Go to the tab "SQL"

  • Post this query in the field :

SELECT id
FROM search_words
WHERE word = 'Belinda'

MySQL returned an empty result set (i.e. zero rows).
SELECT id
FROM search_words
WHERE word = 'Belinda'
LIMIT 0 , 30

That's what I get for ANY search I'm attempting, not only 'Belinda' but even for words that I can indeed find on my forums' search feature.

Vanslyde's Website

11 Reply by Slavok

  • Slavok
  • PunBB Developer
  • Offline
  • From: Russia
  • Registered: 2008-06-11
  • Posts: 1,238

Re: Some posts being ignored when rebuilding the search index?

Sorry, I forgot to warn that the word should be in lower case:

SELECT id
FROM search_words
WHERE word = 'belinda'

Slavok's Website

  • From: Montreal
  • Registered: 2006-10-02
  • Posts: 144

Re: Some posts being ignored when rebuilding the search index?

Slavok wrote:

Sorry, I forgot to warn that the word should be in lower case:

SELECT id
FROM search_words
WHERE word = 'belinda'

Same result: MySQL returned an empty result set (i.e. zero rows)

Vanslyde's Website

13 Reply by Vanslyde (edited by Vanslyde 2009-11-03 06:32)

  • From: Montreal
  • Registered: 2006-10-02
  • Posts: 144

Re: Some posts being ignored when rebuilding the search index?

Vanslyde wrote:
Slavok wrote:

Sorry, I forgot to warn that the word should be in lower case:

SELECT id
FROM search_words
WHERE word = 'belinda'

Same result: MySQL returned an empty result set (i.e. zero rows)

But it's there alright

http://i119.photobucket.com/albums/o144/indiequebec/belinda.gif

I tried

SELECT * FROM `search_words` WHERE word = 'belinda'

And found the Word ID which is 44565.

Now in the "search_matches" table I've got a warning saying

! More than one INDEX key was created for column `word_id`

http://i119.photobucket.com/albums/o144/indiequebec/morethanoneindex.gif

And if I post the query you gave earlier in the SQL tab I get

SELECT *
FROM search_matches
WHERE word_id = 44565

http://i119.photobucket.com/albums/o144/indiequebec/belinda44565.gif

I bet the Warning could have something to do with the problem I'm currently having.
Please continue if you can assist further.

Vanslyde's Website

14 Reply by Slavok

  • Slavok
  • PunBB Developer
  • Offline
  • From: Russia
  • Registered: 2008-06-11
  • Posts: 1,238

Re: Some posts being ignored when rebuilding the search index?

Remove the index for "search_matches" table with "word_id" keyname.
What is the first column in "search_words" table? Post here the structure of "search_words" table, please.

Slavok's Website

  • From: Montreal
  • Registered: 2006-10-02
  • Posts: 144

Re: Some posts being ignored when rebuilding the search index?

Slavok wrote:

Remove the index for "search_matches" table with "word_id" keyname.

done: Index dropped
http://i119.photobucket.com/albums/o144/indiequebec/morethanoneindex2.gif

Slavok wrote:

What is the first column in "search_words" table? Post here the structure of "search_words" table, please.

http://i119.photobucket.com/albums/o144/indiequebec/searchtablestructure.gif

Vanslyde's Website

16 Reply by Slavok

  • Slavok
  • PunBB Developer
  • Offline
  • From: Russia
  • Registered: 2008-06-11
  • Posts: 1,238

Re: Some posts being ignored when rebuilding the search index?

I don't like that all words have id=1. I think it's wrong. Have you installed any mods, which can affected on the search mechanism?

Slavok's Website

  • From: Montreal
  • Registered: 2006-10-02
  • Posts: 144

Re: Some posts being ignored when rebuilding the search index?

Hi again

I'd like to share the resolution for my issue.

The file admin_maintenance.php contained in punbb-1.2.22.zip, was causing these problems when indexing my forums.

I had it replaced with the (newest?) admin_maintenance.php file that says;

Copyright (C) 2002-2005  Rickard Andersson (rickard@punbb.org)

located here: http://fluxbb.org/trac/browser/fluxbb/b … format=txt

All of the posts are now indexed fine : )

Vanslyde's Website

18 Reply by Slavok

  • Slavok
  • PunBB Developer
  • Offline
  • From: Russia
  • Registered: 2008-06-11
  • Posts: 1,238

Re: Some posts being ignored when rebuilding the search index?

Thanks for reporting this. We will investigate the problem.

Slavok's Website