Caching out
September 9th, 2011 at 11:25 pm by Dr. Drang
I can’t say that it’s happened every day, but more days than not over the past week or two, this site has become unavailable. Sometimes I’ve noticed it myself, sometimes a reader has tweeted me about it. I know it’s not because of traffic, but beyond that I have no idea of the cause. I haven’t asked my web host about it because outages of half an hour or so have been common over the past several months, and the host’s support team has never been any help.
It’s possible that some of the problem is due to the WP Super Cache plugin I installed back in early spring. It is, I think, the reason the site logs show a lot of calls to wp-cron.php, which may be causing the brief outages. As I say, I really don’t know if WP Super Cache is the culprit, but I decided to remove it to test whether the site is more reliable without it.
For a brief time I had another caching plugin, W3 Total Cache, installed. Unlike the last time I tried to use it, this installation of W3 Total Cache didn’t immediately crash the blog, but some combination of its settings was making certain posts behave badly, and it kept telling me that the Apache server didn’t have ReWriting enabled, which I knew was wrong. So I removed it, too.
So now it’s running WordPress au naturel, and I’ll be looking to see if that puts a stop to the brief outages. If it does, I’ll keep running without a cache and accept the fact that a significant traffic spike could take he site down. If it doesn’t, I’ll have to start looking for a new host when my current prepaid year is up.
Update 9/12/11
This afternoon’s minor Fireballing has added a factor to the experiment that I wasn’t planning on, but it’s clear from the comments and tweets I got before then that the site is having uptime problems even without WP Super Cache. I may go back to it or test other caching systems. There are suggestions in the comments that I’ll be looking into.
Update 9/13/11
I installed the Hyper Cache plugin late last night to give it a trial. It was very easy to configure and hasn’t caused any unique problems. The site is still having periodic response problems, but they don’t seem related to any particular plugin or set of plugins.
September 10th, 2011 at 7:22 pm
You describe exactly the same issues I’ve been experiencing with my blogs recently. I suspected, by trial and error, that the W3 and WP Super Cache plugins were the issue a week ago. Another blog didn’t have either, so I added one to it and the blog borked, corrupting the .htaccess file. Have since removed both plugins from all of them and so far, so good. Whew!
September 12th, 2011 at 11:35 am
This page just took 3 minutes to fully load.
September 12th, 2011 at 1:57 pm
We experienced a number of issues with W3 Total Cache and eventually removed it in favor of WP Super Cache—so far that has handled our traffic adequately.
If you’re on a shared host then there’s probably not a whole lot you can do, but if you have root access you can install Varnish or a similar HTTP accelerator in front of your web server to relieve the stress on your WordPress installation. (Actually, you can run Varnish on a small Rackspace Cloud Server or Amazon EC2 instance and achieve the same thing. Might be worth investigating.)
September 12th, 2011 at 1:59 pm
well all your rss feeds are being pulled from the slowest service yahoo owns - yahoo pipes, so i’m sure that comes into play, slo you could definitely minify your css/js (as you’ve got a lot of scripts loading at the start and end of each page).
Also yeah, it took around 2-3 min. to load though you’re getting a 30-40ms average ping from my computer but with a 30%+ packet loss…
also their latest(A2WEBHOSTING.COM) tweets were about problems with their hosting and that they had to do a hard reset.. i’m sure they prob still haven’t stopped the trouble maker’s script which could easily slow down the rest of the shared hosting server…..
September 12th, 2011 at 2:04 pm
@sally — Same here on one of the higher traffic blogs that I host. It’s fairly regular.
That coincides with WordPress spawning a ton of processes as the server heats up.
September 12th, 2011 at 2:43 pm
Let me know if you’d like to revisit this in the future. I’d be happy to take a look and get things configured so everything works and cron doesn’t melt your site.
September 12th, 2011 at 3:07 pm
Loading fine for me, despite comments on twitter. Makes me feel better about WP Supercache.
September 12th, 2011 at 3:12 pm
Calls to wp-cron.php are a normal part of WordPress. I haven’t seen a problem where Supercache causes extra calls to wp-cron. Did you set the expiry time really long, just in case your server was “cleaning up” stale cache files too often?
Oh yeah, install the “Subscribe to comments” plugin. It’ll make conversation on your posts so much easier!
September 12th, 2011 at 3:25 pm
Have you thought about trying CloudFlare (https://www.cloudflare.com/overview.html)?
September 12th, 2011 at 3:27 pm
This is a good (aging) write-up on the various WordPress caching options: (http://www.tutorial9.net/tutorials/web-tutorials/wordpress-caching-whats-the-best-caching-plugin/). I’ve had good luck with Hypercache
September 12th, 2011 at 3:31 pm
We run WP Super Cache on 365tomorrows.com and haven’t had any trouble with it, even under steady load. We’ve been linked from BoingBoing a couple of times without issue, and while we haven’t been (Daring) Fireballed, we haven’t seen any cracks in the caching.
Prior to implementing caching we would break the WordPress MySQL connection fairly consistently.
September 12th, 2011 at 4:21 pm
You said that outages have been the norm with your hosting provider - sounds familiar to me. Hey, wait a second…I’m on A2hosting as well! I’m definitely considering getting off, they’ve had problems lately and have changed several settings without telling us.
September 12th, 2011 at 4:33 pm
Brent,
My feeds aren’t coming through Pipes. Look more carefully at the HTML and you’ll see that those
<link>s are commented out.Andrew,
Thanks for the offer of help. I may take you up on it when I have the time to learn.
Donccha,
I tweaked my
.htaccessto set a long expiry time a couple of days ago. It’s also supposed to be gzipping many of my files. I’m leery of adding new plugins at the moment, but subscribing to comments does seem like a good idea.Everyone else,
Thanks for the comments and suggestions. I’ll be following up on your suggestions as time permits.
September 12th, 2011 at 7:18 pm
As we setup WordPress sites for a living, we have various experiences on this matter.
An effective solution is to use Cloudflare + a caching plugin (SuperCache, Total Cache or whatever works for your setup.) Cloudflare would be your first line of protection, whatever comes through will be handled by the plugin.
September 12th, 2011 at 8:34 pm
I used to use WP Super Cache until I found it wasn’t working with a recent WordPress update. At that point I switched to Quick Cache, and I highly recommend it. Easier to configure and it seems to work flawlessly. (Then again, I haven’t been “fireballed” so it hasn’t really been put to the test in my own installation.)
September 13th, 2011 at 12:38 am
It sounds like you are on shared hosting? If so, about all you can do is experiment with caching plugins. Try using something like apache bench (ab on OS X) to see what impact your settings make.
If you are on a virtual server, first thing I’d do is set MaxClients for Apache to some reasonable number. This will help limit the amount of memory Apache consumes. I’d guess 4-6x the number of CPU cores you have available to you is a reasonable upper limit, but also make sure that you still have plenty of RAM available for mySQL and caching data. It looks like that when WordPress is under load on my server, each apache worker uses about 12MB of RAM (most people come up with an estimate that is 2x that because they don’t understand how shared memory works on Linux). With MaxChild set to 16, apache could use around 192MB. On a 512MB VPS, that still leaves a lot of memory free for mySQL and the RAM-based disk cache.
While you are in there, set the timeout for KeepAlives to a small number, like two seconds, or disable them completely. This makes more efficient use of the Apache workers because they won’t be waiting around doing nothing while tying up memory waiting to see if the remote browsers decide to request another page.
Next I’d install a PHP opcode cache, like APC. PHP wastes a lot of time loading and parsing the wordpress code for every page request, opcode caches help reduce this wasted processing and IO.
If you want to get really fancy, set nginx up as a reverse proxy to Apache, and make sure it is set up to buffer results. Nginx only uses a little memory to serve hundreds of concurrent browser requests. When set up as a reverse proxy, it allows apache to move on to a new request quickly while nginx feeds the results back to the browser, which also makes more efficient use of the memory allocated to Apache.
Really though, it sounds like your shared host may be the cause of your reliability problems.
September 13th, 2011 at 12:51 am
Check out Cloudflare.com
Daniel
September 13th, 2011 at 1:55 am
Ehh.. just switch to linode…. install wordpress + follow their guide to make it secure.. then their other guide to streamline wordpress/mysql… it’ll run fast. this isn’t know as being one of the more reliable hosts in terms of access times/speeds.. fyi
September 13th, 2011 at 5:11 am
I can vouch for Linode - I always had flawless performance and uptime from them. However, I am now using Media Temple’s Grid Service in an attempt to avoid anything time consuming in the setup and maintenance of my blog - kids do that to to ya ;)
September 13th, 2011 at 11:33 am
Yeah, I’m a linode user. Performance and uptime have both been awesome, but you do have to admin the server.
September 14th, 2011 at 11:35 am
If you have VPS, you should check your Apache/PHP installation. I was having performance issues with wordpress even with all kinds of caching plugins installed. But then I installed eAccelerator (or xCache) modules on Apache, made some .htaccess changes for browser caching, compression, ET tags etc and the site was almost 60% faster. I then disabled and uninstalled the WP caching plugins and the site remained fast. Thanks Mukesh
September 14th, 2011 at 4:40 pm
I appreciate the time and thought some of you have put into your comments. I should have mentioned in the post that I’m using shared hosting.