Australian Joomla! Community

I have just rebuilt this site, partly in the hopes that eliminating the kunena/jomsocial pair would reduce CPU usage. Unfortunately CPU usage has increased.

Any experts out there with tips? Really not my speciality at all.

Answers on a postcard, quick before they suspend us.
Ta
Dex

Difficult to say, Dex, but it's my personal opinion (and from seeing scores of issues that I've seen reported with the bloatware that comes with JoomlArt templates) that we could easily eliminate the template as the culprit with a bit of harmless experimentation/"basic" Joomla. I know, Beez20 is not our favourite cup of tea/coffee but at least we know that it's lightweight. That might be a good start.

Apart from that, you might find this article worthwhile: Improving Kunena performance

I don't think it is the template framework itself but you can start by cache it up

T3 has a cache within the template.

Join all the CSS files and minify the style sheets.
Join the JS files (can break functionality depending on amount of JS being included)

Turn on JAT3 cache.

That should take care of the T3 side of things.

JomSocial
I know there is a mailing function on there that should be setup as a CRON and not as a web process for sending out mail to and between members.
Check that setting because we've had issues with that in regards to performance and had to tweak it down only to not be able to send emails. So CRON was the solution even though CRONs are memory hogs too.

Jomsocial was using cron, but we have now removed it from the site.

And it defo isnt the template framework, otherwise other sites I'm running would also be having issues.

Also minifying css and js has in my experience never worked on any site with a few extensions running. I have used many different methods, starting with the template's own compression, but it always breaks the site. However, I have maxed up the caching, so you can let me know. This site is fairly extension-free

Hey Dex,

Obvious thing, but have you got Gzip Page Compression [Yes] set in J admin?

Use T3 to at least compress the css and put as a single file. "Scripts Down" extension if not already using.

Remove Opensearch from front page or change to not being opensearch throughtout the site - this is a hog if trying to improve performance. Just change to normal search and see the difference or disable the module on the frontpage to see difference on front compared to other pages.

Also, disable mega-menu and moot-tools . . . unless you are going to use them. Also, do enable cache if not already turned on.

That's my 2 cents to this thread - hopefully something I've said may make a difference

Also: sometimes the server CPU usage is displayed as a percentage, but only based off one CPU. So if you have 8 or 16 CPUs and it's showing 120% that is only 1.2 CPUs out of however many there are. If it's only one CPU you're in trouble, if 2 your "OK", if more than 2 CPUs don't stress - all is good

Max

Gzip ... hmm ... interesting thought.

I agree about the Opensearch and getting rid of mega-menu. A differenent Joomla site template will establish if this is a factor or not.

Mootools is necessary for the forum; it's enabled by default in J! 2.5 anyway and most Joomla sites won't work without it. jQuery can cause Javascript library conflicts but, because Javascript is client-side, these things shouldn't affect site performance.

Server page caching is not worth the effort in sites where the content is constantly changing; caching is mainly used on sites with static content. Caching of SQL queries, however, is a different proposition. Kunena 1.7 doesn't cache SQL queries but K 2.0 does. And, besides, turning on server caching makes a total mess of when you read the Recent Topics; with caching on, after you've read a topic, the "NEW" indicator still shows the topic as unread, unless you refresh the page.

You can disable Kunena's minified CSS if you run Kunena with its "debug mode" enabled. See Minified CSS

My best guess is that either the Latest Post module is ineffiiently querying the DB or that Someone Out There is leeching our capacity.

If it was a general template or caching or Kunena problem, everyone would be getting it, right?

But I did notice that with another J2.5 site, when I put two blogs on one page, the system nearly died with the general overburdenment of DB queries supporting them.The article model is not perhaps as efficient as it could be. And the current blog + Latest Posts is equivalent to two blogs (in terms of DB queries). So perhaps if I get rid of the lastest posts and re-test and then if it's a problem, prune the forum

you know Nick, Sydney Joomla User Group Nick made a plugin to solve just these problems. I was testing it the other day on one of our websites.

conquerjoomla.com

That will help you pin point the prob.

The the params for the plugin turn on the debugging features and then debugger for the website. All the processes on the website are show with time to generate or memory used to generate (one of the two I can't remember now). You'll be able to pinpoint what is taking the most time and kill it.

www.conquerjoomla.com/cjspeed.html

here feature number 1 of CJspeed

Hmm . . . didn't realise kunena needed mootools . . . learn something new every day

What you say may be right Dex, with the main page bringing up the big lots of queries (and the front page seems to be the main offerndor/entry point).

You could list latest Kunena topics just from the "Recent Topics" RSS, rather than querying the forum DB - but only lists most recent topics, not most recent posts . . . still - it would reduce load from DB as is only pulling text from the RSS.

You could looking at the server stats and error log to see of anyone is leeching/loading the system - but, this can be a painful experience, if not enlightening.

Are you getting issues with the forum pages as well?

-> Also you can exclude T3 cache by component, page, etc in the template area . . . so will work for static pages, but can exclude Kunena for example.

dex wrote: Latest posts queries only occur on the page(s) where you display the module. But, it may depend on what version(s) of things you're using.

Would you like to post your config. report at the Kunena website where I can have a look at it for you?

MaxTech wrote:Joomla! will not run without Javascript - at least some form of Javascript library in place. It's that simple. Some Joomla extension developers use jQuery but most extension developers these days rely on Mootools. There was a big discussion (and some debate) about this a while ago as far as the Kunena project was concerned: see MooTools and Kunena 1.6

Thia widespread reliance on Mootools could all change in J! 3.0 where, I understand, there's a move to head towards jQuery/.

Anyway, as I wrote before, Javascript should not be a factor in site performance.

Thanks for all your ideas, people.

Page load 7.9 secs
I tried the latest posts as RSS instead - slowed down page load by 2.5 seconds to 10.5 secs so prob not a winner
I turned off latest posts, speeded page up to 6.9

Tried running CJspeed and hit a speed bump straight away:
Fatal error: Maximum execution time of 30 seconds exceeded in cjspeed/minify/JSMin.php on line 249

Then I stopped it minifying, but didnt really find out much that looked useful

I also cleared the redirect table, which was 10Mb, and deleted all the old 1.5 tables.

i am going now to:
(a) put the cache back on
(b) load a new ultralight template over the weekend

And now Kunena cant be the culprit as there is no Kunena on the home page.

Work continues....

I should also run a link checker and check Google webmaster - seems to be a lot of lost URLs and why are they all popping up right now, when I only just cleared them? Who the hell is visiting all those dead-url pages so quickly???

OK dead links in the move:

1. Anything with 'communityforum' - hence my .htaccess request in a previous post


2. Anything with 'Joomladay.org.au' - I've changed the redirection and filtering to sort this


3. Personal URLs from jomsocial - cant rescue
4. Any article or forum post link - now contain the article/post number, so the SEF URL has changed Google doesnt stand a chance
5. Anything with edit-signature from jomsocial - cant rescue. I'm baffled that there are so many of these coming up

Ok, I removed almost all the features of the site in order to detect the CPU usage bad boy. CPU usage is now very low.
So I have re-enabled the forum to see if that is the culprit.
More news tomorrow.
I will restore the original site as soon as permissable.
Thanks for bearing with me, I'm doing all I can to appease the hosting gods.

You're doing a great job, Dex. Thanks very much for all your (unpaid) work!

I agree. If there is some way that I can be of assistance, please drop me a line. I have a few ideas that might help but I'm largely working in an information vacuum about what "else" is running on this site that is causing (in particular) the slow-down in generating forum pages. The page load times - the information shown at the bottom of these pages - is a good first indicator of performance problems and, in the case of this site, is uncomfortably high given a relatively low numbers of posts (< 7K). Even in larger forums (50K+ postings) the page load times experienced at this site would be a cause for concern.

Compare, for example, the page load times here, with the page load times at these sites:

redcomponent.com/forum/index (121K users, 80K messages)
www.autotreffen.at/forum.html (14K users, 217K messages)
www.ijoomla.com/index.php?option=com_kun...w=listcat&Itemid=112 (103K users, 23K messages)

Does anyone think the CPU culprit could be:

(a) Google and its resource-sucking bot chums?

(b) A hacker leeching our CPU time?

When i trimmed the site back to bare bones (minus the forum and all other add-on extensions) I got vastly improved CPU stats:

Jun 17 2012 (partial stats, 06 hours into the day):
joomla.org.au 0.42 1.03 0.0
Top Process %CPU 27.0 [php]
Top Process %CPU 0.3 httpd [joomla.org.au] [/administrator/templates/bluestork/images/menu/icon-16-mess]
Top Process %CPU 0.1 httpd [joomla.org.au] [/templates/system/css/error.css]

See - no reference to index.php at all. And look at those tiny little usages.

Please note that at the same time I turned everything else off, and installed a miniscule template, I also temporarily blocked all Bots.
And left caching on.

So if it isnt the forum, I'm running out of ideas again. Not getting much illumination from Kunena support on the subj.

I'm starting to feel like something is draining the lifeblood from the site.... and mopping the floor isn't helping.

dex wrote: I have offered to discuss this with you one-on-one but, as there is no PM facility at this site, it's difficult. You could be right; it could be the forum that's the culprit. From what I can tell, the forum configuration parameters used on this site are little changed from when (no doubt) the forum was based on Fireboard. I would like to help (telephone or Skype) at your convenience.

You probably know about as much (or more) about Joomla as I do, but maybe I could trigger a thought that would assist. It's all up to you, Dex.

Sorry I didnt mean you. I guess I thought there'd be a common reason. Several kind and helpful people on Kunena, including you, but nothing that resolves the issue so far.

Just doing some more work now - could chat later?

Latest stats:

Jun 16 2012: (final stats)
joomla.org.au 4.82 1.22 0.0
Top Process %CPU 149 /usr/bin/php /index.php
Top Process %CPU 126 /usr/bin/php /index.php
Top Process %CPU 124 /usr/bin/php /index.php

Jun 17 2012:
joomla.org.au 1.49 1.00 0.0
Top Process %CPU 93.0 /usr/bin/php /index.php
Top Process %CPU 56.5 /usr/bin/php /index.php
Top Process %CPU 55.0 [php]

At last we're heading in the right direction.

Tonight I have upped the caching, expires headers and minifying. I will see tmrw if that has helped.
Having to load every last feature of mootools extended and jquery and bootstrap is not ideal.

Old page load size/time:
All 1Mb 11.67s
CSS 208K 2.55s
JS 708k 6.36s

Now load size/times:
All 827 11.25s
CSS 152k 2.58s
JS 550k 5.42s

Hopefully the expires headers will help, altho they don't seem to work for JS.

Then I'm going to focus on MySQL queries.