Author |
Message |
swampie Board Member
Joined: 11 Feb 2009
Posts: 2
|
Posted: Wed Feb 11, 2009 5:24 pm Post subject: posting problem |
|
|
Hi all when posts are entered on our site we get the following error.
Could not obtain common word list
DEBUG MODE
SELECT m.word_id FROM phpbb_search_wordmatch m, phpbb_search_wordlist w WHERE w.word_text IN ('129304', 'foook', 'honeybee', 'pml', 'testing') AND m.word_id = w.word_id GROUP BY m.word_id HAVING COUNT(m.word_id) > 49357
Line : 309
File : functions_search.php
Would anyone know how we could go about rectifying this problem? I would really appreciate your help as we have been really screwed over by someone who claimed to be the owner of a website building host company but now turns out he was not the owner just an employee and it has now blown up and we are left dangling slightly. |
|
Back to top |
|
|
drathbun Board Member
Joined: 24 Jul 2008
Posts: 729 Location: Texas
|
Posted: Wed Feb 11, 2009 5:30 pm Post subject: Re: posting problem |
|
|
Hi, swampie, and welcome. To be honest, you can remove the common word processing part of the posting process as it doesn't work anyway. It will save some complex queries on your board.
Are you comfortable editing code? _________________ phpBBDoctor Blog |
|
Back to top |
|
|
roadhog Board Member
Joined: 18 Nov 2008
Posts: 96 Location: Central Texas
|
Posted: Wed Feb 11, 2009 5:42 pm Post subject: Re: posting problem |
|
|
If you don't want to edit out the search code, you might check to see if your phpbb_search_wordmatch, and phpbb_search_wordlist tables actually exist in the database, or if the data are corrupted, or missing. |
|
Back to top |
|
|
swampie Board Member
Joined: 11 Feb 2009
Posts: 2
|
Posted: Wed Feb 11, 2009 7:04 pm Post subject: Re: posting problem |
|
|
Ok guys i will be perfectly honest i dont know anything about code or such like. There is one part of the admin panel we currently cant access which is something like phpBB MySQLadmin.
I guess this is the section we need access to?? |
|
Back to top |
|
|
roadhog Board Member
Joined: 18 Nov 2008
Posts: 96 Location: Central Texas
|
Posted: Wed Feb 11, 2009 11:36 pm Post subject: Re: posting problem |
|
|
Access to phpMyAdmin will allow you to see what's actually in the database, and service the data if needed. Your host will need to provide you with a username and password that will allow you to log in, in order to do that, (assuming that access to the database through phyMyAdmin is part of the hosting plan). Some plans don't allow it, which can make life pretty tough, when something goes wrong with your board. |
|
Back to top |
|
|
drathbun Board Member
Joined: 24 Jul 2008
Posts: 729 Location: Texas
|
Posted: Wed Feb 11, 2009 11:46 pm Post subject: Re: posting problem |
|
|
I assume you have ftp access? or someone does? Someone had to be able to upload the files to create the board... _________________ phpBBDoctor Blog |
|
Back to top |
|
|
Sylver Cheetah 53 Board Member
Joined: 17 Dec 2008
Posts: 426 Location: Milky Way
|
Posted: Thu Feb 12, 2009 9:35 am Post subject: Re: posting problem |
|
|
drathbun wrote: | Hi, swampie, and welcome. To be honest, you can remove the common word processing part of the posting process as it doesn't work anyway. It will save some complex queries on your board. |
How to do this? _________________ Image link
My Forum || My Blog
phpBB2 forever! |
|
Back to top |
|
|
drathbun Board Member
Joined: 24 Jul 2008
Posts: 729 Location: Texas
|
Posted: Thu Feb 12, 2009 3:46 pm Post subject: Re: posting problem |
|
|
open includes/functions_search.php
find this code:
Code: | function remove_common($mode, $fraction, $word_id_list = array())
{
global $db; |
After, add
That will short-circuit the remove common logic. If you wanted to still mark words as common words, I would set this up as a nightly batch (cron) job instead. _________________ phpBBDoctor Blog |
|
Back to top |
|
|
Sylver Cheetah 53 Board Member
Joined: 17 Dec 2008
Posts: 426 Location: Milky Way
|
Posted: Fri Feb 13, 2009 5:01 am Post subject: Re: posting problem |
|
|
I am not sure what is with this common words and how to mark and what is it good for, but if you say it's not even working, then I can't miss it.
So if I do that, the search will work a bit faster? I hope I got it right. _________________ Image link
My Forum || My Blog
phpBB2 forever! |
|
Back to top |
|
|
drathbun Board Member
Joined: 24 Jul 2008
Posts: 729 Location: Texas
|
Posted: Fri Feb 13, 2009 1:03 pm Post subject: Re: posting problem |
|
|
Search won't change a bit. Posting speed will improve.
The theory is that common words add no value to the search, and therefore each time someone posts their words are examined to see if they have crossed the "common" percentage threshold. If they have, they will be marked as common and ignored in future search requests. In a default phpbb2 installation, a word that appears in more than 40% of the posts is to be marked common. So first you have to ask yourself, how often will that happen? Remember that common words like "the" are already skipped because of the stop-word process.
Next, here's the query that was posted earlier:
Code: | SELECT m.word_id FROM phpbb_search_wordmatch m, phpbb_search_wordlist w WHERE w.word_text IN ('129304', 'foook', 'honeybee', 'pml', 'testing') AND m.word_id = w.word_id GROUP BY m.word_id HAVING COUNT(m.word_id) > 49357 |
In this case, the number 49357 was determined to be the boundary for 40% of the posts. The point of this query is to find out if any of the words appear in more posts than that. The cost of the query is quite high, as there is a join to what is typically the largest table in the database (search_wordmatch) and an aggregate function count.
I have a board that recently crossed 500,000 posts. In order to be classified as "common" a word would have to exist in over 200,000 posts. How likely is that?
I ran a query to find the 25 most common words used on my board, and it took over a minute to run just that one query. The most common word was used in less than 40,000 posts. That's far less than the 200,000 post threshold required to be marked common. That's why I suggest that the process doesn't work.
In my opinion, it's far more effective to short-circuit the "remove common" process using the technique I already posted, and then to periodically (say every six months or so) do some manual analysis as to which words are becoming common. Adding those common words to the stop list is more effective, and the posting process speed improves. _________________ phpBBDoctor Blog |
|
Back to top |
|
|
Sylver Cheetah 53 Board Member
Joined: 17 Dec 2008
Posts: 426 Location: Milky Way
|
Posted: Fri Feb 13, 2009 2:39 pm Post subject: Re: posting problem |
|
|
Okay. Then I will do that change in functions_search.php. Thanks! _________________ Image link
My Forum || My Blog
phpBB2 forever! |
|
Back to top |
|
|
|