Caching out

I can’t say that it’s happened every day, but more days than not over the past week or two, this site has become unavailable. Sometimes I’ve noticed it myself, sometimes a reader has tweeted me about it. I know it’s not because of traffic, but beyond that I have no idea of the cause. I haven’t asked my web host about it because outages of half an hour or so have been common over the past several months, and the host’s support team has never been any help.

It’s possible that some of the problem is due to the WP Super Cache plugin I installed back in early spring. It is, I think, the reason the site logs show a lot of calls to wp-cron.php, which may be causing the brief outages. As I say, I really don’t know if WP Super Cache is the culprit, but I decided to remove it to test whether the site is more reliable without it.

For a brief time I had another caching plugin, W3 Total Cache, installed. Unlike the last time I tried to use it, this installation of W3 Total Cache didn’t immediately crash the blog, but some combination of its settings was making certain posts behave badly, and it kept telling me that the Apache server didn’t have ReWriting enabled, which I knew was wrong. So I removed it, too.

So now it’s running WordPress au naturel, and I’ll be looking to see if that puts a stop to the brief outages. If it does, I’ll keep running without a cache and accept the fact that a significant traffic spike could take he site down. If it doesn’t, I’ll have to start looking for a new host when my current prepaid year is up.

Update 9/12/11
This afternoon’s minor Fireballing has added a factor to the experiment that I wasn’t planning on, but it’s clear from the comments and tweets I got before then that the site is having uptime problems even without WP Super Cache. I may go back to it or test other caching systems. There are suggestions in the comments that I’ll be looking into.

Update 9/13/11
I installed the Hyper Cache plugin late last night to give it a trial. It was very easy to configure and hasn’t caused any unique problems. The site is still having periodic response problems, but they don’t seem related to any particular plugin or set of plugins.


22 Responses to “Caching out”

  1. sally says:

    You describe exactly the same issues I’ve been experiencing with my blogs recently. I suspected, by trial and error, that the W3 and WP Super Cache plugins were the issue a week ago. Another blog didn’t have either, so I added one to it and the blog borked, corrupting the .htaccess file. Have since removed both plugins from all of them and so far, so good. Whew!

  2. Russell says:

    This page just took 3 minutes to fully load.

  3. andreslucero says:

    We experienced a number of issues with W3 Total Cache and eventually removed it in favor of WP Super Cache—so far that has handled our traffic adequately.

    If you’re on a shared host then there’s probably not a whole lot you can do, but if you have root access you can install Varnish or a similar HTTP accelerator in front of your web server to relieve the stress on your WordPress installation. (Actually, you can run Varnish on a small Rackspace Cloud Server or Amazon EC2 instance and achieve the same thing. Might be worth investigating.)

  4. Brent says:

    well all your rss feeds are being pulled from the slowest service yahoo owns - yahoo pipes, so i’m sure that comes into play, slo you could definitely minify your css/js (as you’ve got a lot of scripts loading at the start and end of each page).

    Also yeah, it took around 2-3 min. to load though you’re getting a 30-40ms average ping from my computer but with a 30%+ packet loss…

    also their latest(A2WEBHOSTING.COM) tweets were about problems with their hosting and that they had to do a hard reset.. i’m sure they prob still haven’t stopped the trouble maker’s script which could easily slow down the rest of the shared hosting server…..

  5. Than says:

    @sally — Same here on one of the higher traffic blogs that I host. It’s fairly regular.

    That coincides with WordPress spawning a ton of processes as the server heats up.

  6. Andrew Nacin says:

    Let me know if you’d like to revisit this in the future. I’d be happy to take a look and get things configured so everything works and cron doesn’t melt your site.

  7. Clark says:

    Loading fine for me, despite comments on twitter. Makes me feel better about WP Supercache.

  8. Donncha O Caoimh says:

    Calls to wp-cron.php are a normal part of WordPress. I haven’t seen a problem where Supercache causes extra calls to wp-cron. Did you set the expiry time really long, just in case your server was “cleaning up” stale cache files too often?

    Oh yeah, install the “Subscribe to comments” plugin. It’ll make conversation on your posts so much easier!

  9. Jim says:

    Have you thought about trying CloudFlare (https://www.cloudflare.com/overview.html)?

  10. Jason Simanek says:

    This is a good (aging) write-up on the various WordPress caching options: (http://www.tutorial9.net/tutorials/web-tutorials/wordpress-caching-whats-the-best-caching-plugin/). I’ve had good luck with Hypercache

  11. Stephen R. Smith says:

    We run WP Super Cache on 365tomorrows.com and haven’t had any trouble with it, even under steady load. We’ve been linked from BoingBoing a couple of times without issue, and while we haven’t been (Daring) Fireballed, we haven’t seen any cracks in the caching.

    Prior to implementing caching we would break the WordPress MySQL connection fairly consistently.

  12. Sam Greene says:

    You said that outages have been the norm with your hosting provider - sounds familiar to me. Hey, wait a second…I’m on A2hosting as well! I’m definitely considering getting off, they’ve had problems lately and have changed several settings without telling us.

  13. Dr. Drang says:

    Brent,
    My feeds aren’t coming through Pipes. Look more carefully at the HTML and you’ll see that those <link>s are commented out.

    Andrew,
    Thanks for the offer of help. I may take you up on it when I have the time to learn.

    Donccha,
    I tweaked my .htaccess to set a long expiry time a couple of days ago. It’s also supposed to be gzipping many of my files. I’m leery of adding new plugins at the moment, but subscribing to comments does seem like a good idea.

    Everyone else,
    Thanks for the comments and suggestions. I’ll be following up on your suggestions as time permits.

  14. FreelanceU says:

    As we setup WordPress sites for a living, we have various experiences on this matter.

    An effective solution is to use Cloudflare + a caching plugin (SuperCache, Total Cache or whatever works for your setup.) Cloudflare would be your first line of protection, whatever comes through will be handled by the plugin.

  15. room34 says:

    I used to use WP Super Cache until I found it wasn’t working with a recent WordPress update. At that point I switched to Quick Cache, and I highly recommend it. Easier to configure and it seems to work flawlessly. (Then again, I haven’t been “fireballed” so it hasn’t really been put to the test in my own installation.)

  16. eas says:

    It sounds like you are on shared hosting? If so, about all you can do is experiment with caching plugins. Try using something like apache bench (ab on OS X) to see what impact your settings make.

    If you are on a virtual server, first thing I’d do is set MaxClients for Apache to some reasonable number. This will help limit the amount of memory Apache consumes. I’d guess 4-6x the number of CPU cores you have available to you is a reasonable upper limit, but also make sure that you still have plenty of RAM available for mySQL and caching data. It looks like that when WordPress is under load on my server, each apache worker uses about 12MB of RAM (most people come up with an estimate that is 2x that because they don’t understand how shared memory works on Linux). With MaxChild set to 16, apache could use around 192MB. On a 512MB VPS, that still leaves a lot of memory free for mySQL and the RAM-based disk cache.

    While you are in there, set the timeout for KeepAlives to a small number, like two seconds, or disable them completely. This makes more efficient use of the Apache workers because they won’t be waiting around doing nothing while tying up memory waiting to see if the remote browsers decide to request another page.

    Next I’d install a PHP opcode cache, like APC. PHP wastes a lot of time loading and parsing the wordpress code for every page request, opcode caches help reduce this wasted processing and IO.

    If you want to get really fancy, set nginx up as a reverse proxy to Apache, and make sure it is set up to buffer results. Nginx only uses a little memory to serve hundreds of concurrent browser requests. When set up as a reverse proxy, it allows apache to move on to a new request quickly while nginx feeds the results back to the browser, which also makes more efficient use of the memory allocated to Apache.

    Really though, it sounds like your shared host may be the cause of your reliability problems.

  17. Daniel says:

    Check out Cloudflare.com

    Daniel

  18. Brent says:

    Ehh.. just switch to linode…. install wordpress + follow their guide to make it secure.. then their other guide to streamline wordpress/mysql… it’ll run fast. this isn’t know as being one of the more reliable hosts in terms of access times/speeds.. fyi

  19. Mike says:

    I can vouch for Linode - I always had flawless performance and uptime from them. However, I am now using Media Temple’s Grid Service in an attempt to avoid anything time consuming in the setup and maintenance of my blog - kids do that to to ya ;)

  20. eas says:

    Yeah, I’m a linode user. Performance and uptime have both been awesome, but you do have to admin the server.

  21. Mukesh says:

    If you have VPS, you should check your Apache/PHP installation. I was having performance issues with wordpress even with all kinds of caching plugins installed. But then I installed eAccelerator (or xCache) modules on Apache, made some .htaccess changes for browser caching, compression, ET tags etc and the site was almost 60% faster. I then disabled and uninstalled the WP caching plugins and the site remained fast. Thanks Mukesh

  22. Dr. Drang says:

    I appreciate the time and thought some of you have put into your comments. I should have mentioned in the post that I’m using shared hosting.