phpBB2Refugees.com Logo
Not affiliated with or endorsed by the phpBB Group

Register •  Login 

Continue the legacy...

Welcome to all phpBB2 Refugees!Wave Smilie

This site is intended to continue support for the legacy 2.x line of the phpBB2 bulletin board package. If you are a fan of phpBB2, please, by all means register, post, and help us out by offering your suggestions. We are primarily a community and support network. Our secondary goal is to provide a phpBB2 MOD Author and Styles area.

Getting too popular for it's own good!
1 members found this topic helpful
Goto page 1, 2, 3, 4  Next
 
Search this topic... | Search phpBB2 Discussion... | Search Box
Register or Login to Post    Index » phpBB2 Discussion  Previous TopicPrint TopicNext Topic
Author Message
Jim_UK
Board Member



Joined: 19 Nov 2008

Posts: 544
Location: North West UK


flag
PostPosted: Tue Feb 03, 2009 10:58 am 
Post subject: Getting too popular for it's own good!

My sons model ship site is probably the most popular English speaking one on the net and steadily growing.
Below is a query posted by one of it's members re searches.
BANYAN wrote:
Folks, our popularity may become the cause of some concern into the future and it may bear some thinking ahead of time on how to cope with it. HOWEVER - IT IS GOOD TO BE POPULAR icon_smile.gif

I already have some problems finding certain topics etc for a detail I remember seeing etc (Manual or SEARCH tool). I also experience the problem of monitoring the growing number of thread topics (many more daily posts these days - or so it seems). Other members (for example Bosco) are also saying they are having difficulty finding some of their previous threads/topics.

It would be easy to say to just cull topics older tha xxxx after the forum gets too big, but we could lose a lot of valuable info in just doing that. If I recall correctly, one of the previous ideas was to use MSD (Model Shipwrights Database)to collect and save a lot of this info.

No matter which way we go, it is going to take a considerable amount of work to prune the valuable stuff from the background chatter in a lot of the topics. Also, some of these topics could be renamed at the same time (cut and diced if necessary) to make searching easier, but we would need to be carful to maintain the owner/author of the associated post while doing this.

How do we go about this in the future? Sorry Jim or Uwe if I am pre-empting something you may already have in mind. I may have some free time to assist with such a task this year, but I would not be able to do the lot no matter what the system/plan we choose to use. I don't think that we can continue to grow at this rate without some pruning of the threads in the future and not lose valuable info?

Over to you blue leader and fellow mods - what are your thoughts?

cheers

Pat


What do others do about the vast numbers of topics that they accumulate? You can see the problem - there is a need to preserve all the material and make it searchable but at the same time pick out the "flesh from the bones" so that the searches get you the info you seek.

An example that someone gave was that they were searching for info on how to make "tree nails" with a "draw plate". A search with those fields produced dozens of hits on different topics over several different forums.

I have suggested to my son that what he should do is to ask the user to go through the topics and pick the meat off and then incorporate it into a tutorial accessed via the Navigation. The first place to seek info should be to look to see if there is a tutorial.

What do you do or has the situation not arisen?

Jim
Back to top
dogs and things
Board Member



Joined: 18 Nov 2008

Posts: 621
Location: Spain


flag
PostPosted: Tue Feb 03, 2009 12:10 pm 
Post subject: Re: Getting too popular for it's own good!

Hi Jim,

In part fortunately I haven't had to face this kind of bigger-board complications yet.

On thing I can think of to narrow down the search results is to exclude the forums with less important content from indexing.

The attached MOD allows you to mark certain forums as not-indexable.
This, together with the Rebuild Search MOD would reduce the number of results and narrow the searches down to the forums witj more relevant content.

Don´t know if this would suit your son's board's needs but mention it just in case...

Greetings.



disable_indexing-1.zip
 Description:
Disable Search Indexing

Download
 Filename:  disable_indexing-1.zip
 Filesize:  24.06 KB
 Downloaded:  592 Time(s)


_________________
phpBB2 will never die, I hope!
Back to top
Jim_UK
Board Member



Joined: 19 Nov 2008

Posts: 544
Location: North West UK


flag
PostPosted: Tue Feb 03, 2009 12:18 pm 
Post subject: Re: Getting too popular for it's own good!

I am not sure but maybe. It certainly sounds useful in any case and I shall add it to the archive of mods that I have for phpBB2 and may even use it on my own to exclude certain forums.

His problem basically is that there is a wealth of info but contained in many different topics. A prune would lose so much and even if there were a more efficient search it would still arrive at the same results - lots of posts spread over many topics and forums all touching on the same item they want info on.

We are probably all in the same "boat" (pun intended) with this and even if the topics were posted in the same forum there would still be x different topics containing useful info and a mass of posts that added nothing to that knowledge. (Off topic and trivial posting)


Jim
Back to top
dogs and things
Board Member



Joined: 18 Nov 2008

Posts: 621
Location: Spain


flag
PostPosted: Tue Feb 03, 2009 12:51 pm 
Post subject: Re: Getting too popular for it's own good!

Remember that if you do use this MOD you would have to empty the seach tables and rebuild the search. From then on the excluded forums wouldn't be indexed anymore.

But somehow I figure the problem would remain more or less the same, too many results in an unordered list.

One thing I started doing on my board is making recopilation topics containing the most important topics regarding specific matter. An example on my dog behavior board is the most consulted thing, house-training. There's loads of topics on this items, too many to find the most relevant ones. We, myself and the moderators, filtered out the most important ones and posted a topic containing their links.

A tedious job, but, if you go little by little dealing with one subject at the time you'll end up with a gradually increasing list of topics containing relevant links to different matter. And that will be very valuable extra content enhancing the board as a source of info.

For a while I used the MOD I posted, but, discovered that users not only search for important info, they also use the search to find "silly" topics/posts for specific reasons. If you exclude the off-topic forum from indexing nobody would be able to find back anything in that forum using the search.

And the standard advanced search form already does allow to specify a certain forum to be searched, thus excluding the rest.

So, after a while I removed the MOD.

As my site not only contains a board but also html pages I wanted a site-wide search tool. I didn´t want to use Google Search and discovered Sphider. Sphider indexes my html pages and my board's content.

This means that I ended up with two search engines on my board. Although the phpBB2 search engine is more useful for detailed searches I like my Sphider search engine better. One good thing about the Sphider search is that it's search results are limited to the relevant forums.

_________________
phpBB2 will never die, I hope!
Back to top
Holger
Board Member



Joined: 19 Jan 2009

Posts: 509
Location: Hanover


flag
PostPosted: Tue Feb 03, 2009 1:58 pm 
Post subject: Re: Getting too popular for it's own good!

I have the same problem ...
We are growing very fast (not THAT big yet, but in our niche we are the biggest) and the search is to no help at all soon.

I would love to see a "search in the searchresults" mod or something.

Also, I would like to have "more" wildcards
"glasstrut" and "glassbåt" is found when searching with "glass*".
"vaniljrån" and "bankrån" is found when searching with "*rån".
"kundrelation", "hund" and "undkoja" is found when searching with "*und*".
Wildcards for one letter: word? (result: "words" but not "wording")
Exact phrases: "phrase"
Exclude words: -word
Search in a specific thread: "Search in THIS thread"
Back to top
Oliver2000
Board Member



Joined: 29 Jan 2009

Posts: 21


flag
PostPosted: Tue Feb 03, 2009 2:00 pm 
Post subject: Re: Getting too popular for it's own good!

Hi,

I acctually really like the idea from "dogs and things" and i am going to log a little bit the next days the most common searchs to create late some topics with the links to the best results and capture this search to display direct the best results without even using the phpbb search.

Somehow i anyway need to do something since my search tables have in the meanwhile around 50.000.000 lines and 1GB in the db. I just wanted to kill the entire searchtables and rebuild but this is going to take a long while. After i start a test with rebuild it looks like 5 days streight rebuilding to get the index work again. (even on quad core with 8gb mem etc. and dedicated sql server for the job)

I would like to get the entire search out if there is a better solution. Does somebody use something else as the build in search engine? Does somebody has maybe a need little mod to make the search lighter and faster for bigger boards?
Back to top
Daz
Board Member



Joined: 17 Dec 2008

Posts: 32
Location: Yorkshire, England


flag
PostPosted: Tue Feb 03, 2009 3:24 pm 
Post subject: Re: Getting too popular for it's own good!

Hi,

I would consider adding the ability to attach a relativity rating to a topic per phrase/word (for users or just moderators/admins/groups) and then order the search result by it. A per user rating might be worth considering - and put the average in the posts table?
Starting off with the logging of searches would make for an easier to achieve gradual process. Add the ability to rate the results for people when they are searching?

You could auto-generate topics based on the most common queries.

Daz
Back to top
drathbun
Board Member



Joined: 24 Jul 2008

Posts: 653
Location: Texas


flag
PostPosted: Tue Feb 03, 2009 4:19 pm 
Post subject: Re: Getting too popular for it's own good!

Search is one of my favorite topics. icon_smile.gif

One thing to check is to see what size words you have in your search words table. The intent from phpBB2 is to only index words between 3 and 20 words long. However, there is a bug in the regex used in some versions that allows longer and shorter words to get indexed. If you check your search words table with this query;
Code:
select length(word_text) as word_length, count(*) as word_count
from phpbb_search_wordlist
group by 1

If you have any words of 1, 2, or 21+ length then you're indexing extra content.

You can also reduce the size of your search word tables by investing heavily in "stop" words, which create a list of words that won't be indexed. Any word that appears in more than 10-15% of your posts is a likely candidate for a stop word.

Ultimately a large search table is not a problem, as long as it's cached and indexed properly.

This board (refugees) has options for "Search This Topic" and "Search This Forum" which I like. (They're both present in phpBB3 as well.) This board also has a topic "points" system, and by default search results are weighted by point total. The intent there, clearly, is to force higher rated topics to bubble to the top of the search results. In theory a user would add a point to a topic they find useful. The reality is that on my larger board less than 2% of the topics get points, and I am not sure it's because people don't know about the feature or just don't care. icon_smile.gif

What is missing (and much requested) is a true phrase search. Being able to use boolean searches like google would be really nice as well.

_________________
phpBBDoctor Blog
Back to top
Holger
Board Member



Joined: 19 Jan 2009

Posts: 509
Location: Hanover


flag
PostPosted: Tue Feb 03, 2009 4:35 pm 
Post subject: Re: Getting too popular for it's own good!

drathbun wrote:
If you have any words of 1, 2, or 21+ length then you're indexing extra content.


1 36
2 615
...
21 31
22 30
23 20
24 15
25 4
27 3
28 2
29 2
30 1
35 1

Where do I find that bug?

Quote:
This board (refugees) has options for "Search This Topic" and "Search This Forum" which I like.

Me too! Me too! Where do me get those?
Back to top
Jim_UK
Board Member



Joined: 19 Nov 2008

Posts: 544
Location: North West UK


flag
PostPosted: Tue Feb 03, 2009 5:58 pm 
Post subject: Re: Getting too popular for it's own good!

Some useful info but sadly not what the topic was about. The problem is what to do when there are so many topics of a similar nature such that a search produces say dozens of hits. All the search hits being legitimate ones we now have a vast amount to read through but as with many topics that can go off course icon_wink.gif only a small section of each topic may contain useful information.

Jim
Back to top
Holger
Board Member



Joined: 19 Jan 2009

Posts: 509
Location: Hanover


flag
PostPosted: Tue Feb 03, 2009 7:22 pm 
Post subject: Re: Getting too popular for it's own good!

Ah! Sorry for going offtopic!

Now I see the big problem.
I think this is a moderation issue.
Same topic, same content = join into one topic.
This is the way I deal with it.
There is no way to solve such a thing with the search-function.
Back to top
Jim_UK
Board Member



Joined: 19 Nov 2008

Posts: 544
Location: North West UK


flag
PostPosted: Tue Feb 03, 2009 7:31 pm 
Post subject: Re: Getting too popular for it's own good!

In the case that I sited the subject of "tree nails" had been raised in several topics but had also been the basis of several topics.
The content of some part of each topic was worth preserving but as usual there would be a lot of trivia added by users.
The search returns lots of valid hits but not all the content is worth reading. Now you can not go through topics telling posters that their input was worthless and contributed nothing to other users education (even if it is true)
That was why I suggested to my son that they have someone go through such topics and create tutorials just as they have with some other aspects of the hobby.
The problem of course is that as the site grows this sort of thing is going to become an ever recurring problem as more and more topics are discussed several times.

Jim
Back to top
drathbun
Board Member



Joined: 24 Jul 2008

Posts: 653
Location: Texas


flag
PostPosted: Tue Feb 03, 2009 7:47 pm 
Post subject: Re: Getting too popular for it's own good!

Jim, have you considered a wiki as an addition to your board? I have, but so far have not managed to set aside the time to integrate it properly. The concept of a wiki is it becomes the repository for information about specific items, and from there you can link to any number of support topics about the page.
_________________
phpBBDoctor Blog
Back to top
Holger
Board Member



Joined: 19 Jan 2009

Posts: 509
Location: Hanover


flag
PostPosted: Tue Feb 03, 2009 7:49 pm 
Post subject: Re: Getting too popular for it's own good!

A wiki or a lexikon: amigalink.de has a great lexicon-mod.
Back to top
Jim_UK
Board Member



Joined: 19 Nov 2008

Posts: 544
Location: North West UK


flag
PostPosted: Tue Feb 03, 2009 7:53 pm 
Post subject: Re: Getting too popular for it's own good!

drathbun wrote:
Jim, have you considered a wiki as an addition to your board? I have, but so far have not managed to set aside the time to integrate it properly. The concept of a wiki is it becomes the repository for information about specific items, and from there you can link to any number of support topics about the page.


Ah!! That sounds like a good idea. Is there a Wiki mod or is it a stand alone.

Holger wrote:
A wiki or a lexikon: amigalink.de has a great lexicon-mod.

Thanks I shall take a look at that one.

Jim
Back to top
Display posts from previous:   
Register or Login to Post    Index » phpBB2 Discussion  Previous TopicPrint TopicNext Topic
Page 1 of 4 All times are GMT
Goto page 1, 2, 3, 4  Next
 
Jump to:  

Index • About • FAQ • Rules • Privacy • Search •  Register •  Login 
Not affiliated with or endorsed by the phpBB Group
Powered by phpBB2 © phpBB Group
Generated in 0.0798 seconds using 17 queries. (SQL 0.0026 Parse 0.0648 Other 0.0125)
phpBB Customizations by the phpBBDoctor.com
Template Design by DeLFlo and MomentsOfLight.com Moments of Light Logo