phpBB2Refugees.com Logo
Not affiliated with or endorsed by the phpBB Group

Register •  Login 

Continue the legacy...

Welcome to all phpBB2 Refugees!Wave Smilie

This site is intended to continue support for the legacy 2.x line of the phpBB2 bulletin board package. If you are a fan of phpBB2, please, by all means register, post, and help us out by offering your suggestions. We are primarily a community and support network. Our secondary goal is to provide a phpBB2 MOD Author and Styles area.

Question about performance improvements to viewtopic.php

Goto page 1, 2  Next
 
Search this topic... | Search phpBB2 Discussion... | Search Box
Register or Login to Post    Index » phpBB2 Discussion  Previous TopicPrint TopicNext Topic
Author Message
nostro
Board Member



Joined: 16 Jul 2012

Posts: 54


flag
PostPosted: Sat Oct 25, 2014 7:34 pm 
Post subject: Question about performance improvements to viewtopic.php

I'm trying to think of ways of making viewtopic load faster (for phpbb2 and the fork too), but I don't know if the effort will justify the effort, maybe someone has tried some of these ideas before and know if they make sense.

1. In viewtopic, especially if configured with more than 10 replies per page, in most pages there is usually more posts than users... I mean, usually there is at least one user that wrote more than 1 of the posts of that page.

Having that in mind, would it make sense if in viewtopic.php after processing each user_sig we keep it so if we need it later we don't have to parse it again? (I have more than one style installed, I can't just cache user's posts and sigs).

2. Should I try to cache parsed (for default style) posts and sigs? At least the page should load faster for guests and users that have selected the default style. Will it deserve the effort? Better to try to use the cache posts mod or to start from scratch?

3. Every post (and subject and signature and polls) is processed in viewtopic (and topic title in viewtopic.php) to censor words (if there is any censored word configured). Maybe there is not too much to gain here, but would be a possibility to check, when you add/edit/remove a word in admin/admin_words.php, what posts will need a word replaced and mark them so then? Would that be a problem for big forums (for example over 1 million posts)? Maybe a cron job when there are too many words and posts?

Any other ways to improve viewtopic speed (*)? I'm thinking of just removing the "posts since x time, order by time desc or asc) form or the jumpbox, since I'm not sure if that many people would miss them.

(*) I mean apart from those mentioned in the "Tweaks for large forums" thread
Back to top
nostro
Board Member



Joined: 16 Jul 2012

Posts: 54


flag
PostPosted: Sun Oct 26, 2014 6:56 am 
Post subject: Re: Question about performance improvements to viewtopic.php

Not sure but I think I read this one in the larger forums thread, maybe not a bad idea...

4. In viewtopic.php, instead of incrementing topic_views field, create a new table and insert a record into that table with the topic_id and then every x minutes (using cron) update topics_table and delete all records of that topic_views_table.

Someone has tried that one?
Back to top
nostro
Board Member



Joined: 16 Jul 2012

Posts: 54


flag
PostPosted: Sun Oct 26, 2014 7:48 am 
Post subject: Re: Question about performance improvements to viewtopic.php

ok, I've implemented #1 because that was easy, but I'm not sure if it loads any faster... looks like it loads a few miliseconds faster (by a few I mean less than 5, that's on a page with 15 posts and 7 different users) ...but it can't do any harm so I'll keep it.


nostro wrote:
http://www.phpbb2refugees.com/viewtopic.php?p=7938#7938
A forum with 175,000 posts:

Viewtopic:
0.059 ms
Back to top
Vendethiel
Board Member



Joined: 26 Oct 2014

Posts: 60



PostPosted: Sun Oct 26, 2014 1:39 pm 
Post subject: Re: Question about performance improvements to viewtopic.php

(I'm sorry it seems like I'm hijacking this topic, but... Which fork?)
Back to top
nostro
Board Member



Joined: 16 Jul 2012

Posts: 54


flag
PostPosted: Sun Oct 26, 2014 2:26 pm 
Post subject: Re: Question about performance improvements to viewtopic.php

The one I'm (slowly) building:
http://www.phpbb2refugees.com/viewtopic.php?t=1187

I'm trying to improve security, clean and make the codebase easily maintainable, I'll remove oop code (because I want the code to be as simple as possible so anyone can work with it, without having to be a php guru), make it faster and lighter, add some important mods to the core (subforums, quick reply, report posts, post aproval...), and use some kind of build system to allow to decide which of those included mods do you want to install, so the rest of the code can be removed.

It will not be newbie admin friendly and old mods and styles won't work.

Probably I'll include a tool to convert from phpbb2, not because I expect many conversions but because I'll need one for my old forums.
Back to top
nostro
Board Member



Joined: 16 Jul 2012

Posts: 54


flag
PostPosted: Mon Oct 27, 2014 8:40 pm 
Post subject: Re: Question about performance improvements to viewtopic.php

Quote:
3. Every post (and subject and signature and polls) is processed in viewtopic (and topic title in viewtopic.php) to censor words (if there is any censored word configured). Maybe there is not too much to gain here, but would be a possibility to check, when you add/edit/remove a word in admin/admin_words.php, what posts will need a word replaced and mark them so then? Would that be a problem for big forums (for example over 1 million posts)? Maybe a cron job when there are too many words and posts?

#3 done, but still not sure if much gained (if any) from that.

nostro wrote:
2. Should I try to cache parsed (for default style) posts and sigs? At least the page should load faster for guests and users that have selected the default style. Will it deserve the effort? Better to try to use the cache posts mod or to start from scratch?

#2 not easy to do, but from the first test it looks like it will be time well spent, it's definitely a bit faster.

With posts (and sigs) cached ...plus caching bbcode_tpl (although caching bbcode_tpl may not be a noticeable improvement, but at least it was not difficult to do).
Quote:
Viewtopic (time - memory - peak memory):
0.047s - 1.43 MiB - 1.89 MiB (if using board default style and all 15 cached posts and sigs can be used)


And this was before making those changes (#1, #2 and #3) (I'm testing the same page).
Quote:
Viewtopic:
0.059s - 1.39 MiB - 1.78 MiB (15 posts per page)


I think the cache posts mod created the cached versions in viewtopic.php and saved them to the filesystem. I'm storing it in the posts_text table, and using a stripped down version of viewtopic in a cron job to generate the cache (will also use posting.php when the user_style is the same as the board default one). Using viewtopic+filesystem may be also a good idea but I want to have control over what posts are cached and when, and be able to easily invalidate the cache.
Back to top
drathbun
Board Member



Joined: 24 Jul 2008

Posts: 653
Location: Texas


flag
PostPosted: Tue Oct 28, 2014 12:49 pm 
Post subject: Re: Question about performance improvements to viewtopic.php

nostro wrote:
1. In viewtopic, especially if configured with more than 10 replies per page, in most pages there is usually more posts than users... I mean, usually there is at least one user that wrote more than 1 of the posts of that page.

Having that in mind, would it make sense if in viewtopic.php after processing each user_sig we keep it so if we need it later we don't have to parse it again? (I have more than one style installed, I can't just cache user's posts and sigs).

This is quite easy to do, is in fact one of the items from phpBB3 that I borrowed and have implemented on this board.

Quote:
2. Should I try to cache parsed (for default style) posts and sigs? At least the page should load faster for guests and users that have selected the default style. Will it deserve the effort? Better to try to use the cache posts mod or to start from scratch?

This is probably not worth it. I leave caching of larger quantities of data up to the web host / operating system. Item 1 isn't really caching, exactly, but avoiding processing the same data over and over on the same page. Deciding which topics / posts are "hot" enough to cache is probably less bang for the buck.

Quote:
3. Every post (and subject and signature and polls) is processed in viewtopic (and topic title in viewtopic.php) to censor words (if there is any censored word configured). Maybe there is not too much to gain here, but would be a possibility to check, when you add/edit/remove a word in admin/admin_words.php, what posts will need a word replaced and mark them so then? Would that be a problem for big forums (for example over 1 million posts)? Maybe a cron job when there are too many words and posts?

I would not do this, personally. As you observe with a large forum the update cycle is going to take a long time to run. Meaning if you decide at some point to add (or remove) a word from a censor list, processing all of the posts is potentially going to be problematic. Plus if you overwrite the original post content you're "editing" what the original poster said. I prefer to leave things as written.

Quote:
Any other ways to improve viewtopic speed (*)? I'm thinking of just removing the "posts since x time, order by time desc or asc) form or the jumpbox, since I'm not sure if that many people would miss them.

Any speed improvements from removing these menu choices would probably be minimal, and if there is even one person that uses them and you remove them you can be sure to hear about it. Ask me how I know. icon_wink.gif

_________________
phpBBDoctor Blog
Back to top
Vendethiel
Board Member



Joined: 26 Oct 2014

Posts: 60



PostPosted: Tue Oct 28, 2014 12:55 pm 
Post subject: Re: Question about performance improvements to viewtopic.php

You might be interested by this, in case you havn't read it yet: https://www.phpbb.com/community/viewtopic.php?f=18&t=135383
Back to top
nostro
Board Member



Joined: 16 Jul 2012

Posts: 54


flag
PostPosted: Tue Oct 28, 2014 4:06 pm 
Post subject: Re: Question about performance improvements to viewtopic.php

drathbun wrote:
Quote:
2. Should I try to cache parsed (for default style) posts and sigs? At least the page should load faster for guests and users that have selected the default style. Will it deserve the effort? Better to try to use the cache posts mod or to start from scratch?

This is probably not worth it. I leave caching of larger quantities of data up to the web host / operating system. Item 1 isn't really caching, exactly, but avoiding processing the same data over and over on the same page. Deciding which topics / posts are "hot" enough to cache is probably less bang for the buck.

I'm still undecided, I'm going to build a test board with several millions of posts and see how it works out. But for now apparently caching parsed posts works, looks like a good idea at least on boards with just one style.

drathbun wrote:
Quote:
3. Every post (and subject and signature and polls) is processed in viewtopic (and topic title in viewtopic.php) to censor words (if there is any censored word configured). Maybe there is not too much to gain here, but would be a possibility to check, when you add/edit/remove a word in admin/admin_words.php, what posts will need a word replaced and mark them so then? Would that be a problem for big forums (for example over 1 million posts)? Maybe a cron job when there are too many words and posts?


I would not do this, personally. As you observe with a large forum the update cycle is going to take a long time to run. Meaning if you decide at some point to add (or remove) a word from a censor list, processing all of the posts is potentially going to be problematic. Plus if you overwrite the original post content you're "editing" what the original poster said. I prefer to leave things as written.

Yes, processing all the posts at once can be a problem for big boards, I'm testing it now, I think letting the task to be done by cron jobs should be good enough, maybe changing one word at a time, or only a word for a subforum, it shouldn't be a problem if it take 100 times or 500 to complete the task, because changing censored words is not something done frequently. The problem is if gain is too small compared to the added complexity.

For the fork maybe I'll leave it as (phpbb2) is now, but adding other mode, one where admin can change the original posts, and posts with censored words are just not allowed to be added to the database.

drathbun wrote:
Quote:
Any other ways to improve viewtopic speed (*)? I'm thinking of just removing the "posts since x time, order by time desc or asc) form or the jumpbox, since I'm not sure if that many people would miss them.

Any speed improvements from removing these menu choices would probably be minimal, and if there is even one person that uses them and you remove them you can be sure to hear about it. Ask me how I know. icon_wink.gif

True for my phpbb2 forum, but for the fork I won't care too much about keeping old "features". But I will probably keep these if removing them doesn't improve anything.

Does phpbb 3.0.x use ajax to load the jumpbox only when someone click on it? or 3.1 do that?
Back to top
drathbun
Board Member



Joined: 24 Jul 2008

Posts: 653
Location: Texas


flag
PostPosted: Wed Oct 29, 2014 10:55 pm 
Post subject: Re: Question about performance improvements to viewtopic.php

The jumpbox can't be cached because it's driven by user permissions. Certain users see some forums and not others. Now as to whether it can be built dynamically (on request) I don't know. You're right in that most folks probably don't even know it's there, so running the query "on demand" via ajax or something similar could save some time.

If I were worried about performance though I would hit the heavy stuff first. There are lots of places to cache data for the board index as an example. You can store the "latest post" and "latest poster" right on the forums table rather than joining to the topics / posts table to pull that information. Sure, it might get out of sync every now and then, but it's close enough.

_________________
phpBBDoctor Blog
Back to top
nostro
Board Member



Joined: 16 Jul 2012

Posts: 54


flag
PostPosted: Thu Oct 30, 2014 7:54 am 
Post subject: Re: Question about performance improvements to viewtopic.php

Not for my forum, but apparently the jumpbox is a problem if there are too many subforums,
http://www.phpbb2refugees.com/viewtopic.php?p=7602

I'll hit the heavy stuff sooner or later (in my forum most of it is already hit), but I think I'll also try to cache the jumpbox (for guests permissions) and use ajax, at least for the fork.
Back to top
nostro
Board Member



Joined: 16 Jul 2012

Posts: 54


flag
PostPosted: Thu Oct 30, 2014 5:19 pm 
Post subject: Re: Question about performance improvements to viewtopic.php

Looks like some time can be saved... for a board with around 40 forums it takes 8-9 ms (3 ms for one with just 4 forums) to generate the jumpbox form (make_jumbox() + parsing template), so I'm doing the ajax thing also for my phpbb2 board.
Back to top
drathbun
Board Member



Joined: 24 Jul 2008

Posts: 653
Location: Texas


flag
PostPosted: Fri Oct 31, 2014 6:41 pm 
Post subject: Re: Question about performance improvements to viewtopic.php

It has been a long time since I have looked at the code, but what I remember is that part of the performance hit is going through the security. You might be better of caching security settings for the user and building the jumpbox using the cached security profile. For this board when you log in your list of authorized forums are cached so any time I do anything related to security (index, viewforum, viewtopic, and so on) I check the cached value. That would save a bunch of queries on a busy board.
_________________
phpBBDoctor Blog
Back to top
nostro
Board Member



Joined: 16 Jul 2012

Posts: 54


flag
PostPosted: Fri Oct 31, 2014 8:09 pm 
Post subject: Re: Question about performance improvements to viewtopic.php

In the make_jumpbox function there is no look to user permissions, by default it just shows every forum that is set to ALL or REG (yeah, that's a bit hackish), so mod/admin/private forums doesn't appear there... but the code for checking forums against user permissions is also there, only that is commented out.

I've completed the ajax thing for my board, in fact I'm using that code that checks user permissions. But I've not done the permission caching part yet, since it loads relatively fast and (I suppose) almost nobody uses the jumpbox... and I know nothing about how permissions work, I should take a look at it.

So you cache the forum permissions for every user, right? Would that be a problem for a board with many users or many forums? Maybe there is usually a relatively low number of different permissions (many of the users will have the same permissions profile), would it be possible to use that or may be too complicated?
Back to top
nostro
Board Member



Joined: 16 Jul 2012

Posts: 54


flag
PostPosted: Fri Oct 31, 2014 8:22 pm 
Post subject: Re: Question about performance improvements to viewtopic.php

Oh, I think I forgot to comment... I also implemented idea #4, and it turned out to be a simple and effective solution.
Back to top
Display posts from previous:   
Register or Login to Post    Index » phpBB2 Discussion  Previous TopicPrint TopicNext Topic
Page 1 of 2 All times are GMT
Goto page 1, 2  Next
 
Jump to:  

Index • About • FAQ • Rules • Privacy • Search •  Register •  Login 
Not affiliated with or endorsed by the phpBB Group
Powered by phpBB2 © phpBB Group
Generated in 0.0153 seconds using 16 queries. (SQL 0.0013 Parse 0.0006 Other 0.0133)
phpBB Customizations by the phpBBDoctor.com
Template Design by DeLFlo and MomentsOfLight.com Moments of Light Logo