Part 1 only really addressed opcode ¹ and Object caching ² and didn’t really touch page caching ³. In this post I have revisited all tests and added in comparisons of using both the APC Object Cache + Batcache plugins as well as using the W3 Total Cache plugin.

Tests

• No opcode, no caching
• APC opcode, no caching
• APC opcode, APC object caching plugin
• APC opcode, W3 Total Cache APC object caching
• APC opcode, APC object caching plugin, Batcache page caching
• APC opcode, W3 Total Cache APC object and page caching

Comparison Stats

• PHP generation time ⁴
• Number of include/include_once/require/require_once calls ⁵
• Number of stat() calls per dtruss/strace ⁶
• cURL time to start transfer ⁷
• Apache Bench (ab) tests for concurrency ⁸ and requests per second

For the above stats gathering, with PHP generation time and cURL time to start transfer, 102 sets were collected, the first 2 were dropped due to cache priming, the remaining 100 were used, and averaged. With the Apache Bench tests, 12 sets were used, dropping the highest and lowest value, and averaging across the remaining 10. Include and stat() counts were gathered over 5 sets not requiring averaging as they were the same between runs.

To find the optimal concurrency and req/s for Apache Bench, I performed manual testing, visually inspecting the results until I reached what I classified as a “sweet spot”. Using the “sweet spot” stats, I performed additional sets to gather the averages for requests per second.

The Setup

• 256MB Rackspace Cloud Server
• Ubuntu 11.04 amd64
• Apache 2.2.17 – Default Ubuntu Install, no modifications, default document root located at /var/www
• PHP 5.3.5 (mod_php) – Default Ubuntu Install, no modifications
• PHP APC 3.1.3p1 – Default Ubuntu Install, no modifications
• MySQL 5.1.54 – Default Ubuntu Install, no modifications
• WordPress 3.3-beta4-r19470 – Default Install, requests made to the “home” page
• APC Object Cache trunk version
• Batcache trunk version
• W3 Total Cache 0.9.2.4

I have not compared static file caching yet and hope to compare W3 Total Cache and WP Super Cache in the future.  In this comparison I am mainly focusing on opcode, object caching and page caching.

I am going to try to keep this comparison about the stats only, and not make this a critique or review of the plugin, although in some cases this will not be possible.

Test Data

No opcode and no caching:
PHP Generation Time: 0.13787 seconds
Number of includes: 80
Number of stat calls: 266
cURL time to start transfer: 0.15463 seconds
Apache Bench Concurrency: 15
Apache Bench Requests Per Second: 19.1483 req/s

APC opcode and no caching:
PHP Generation Time: 0.05088 seconds
Number of includes: 80
Number of stat calls: 148
cURL time to start transfer: 0.05673 seconds
Apache Bench Concurrency: 60
Apache Bench Requests Per Second: 68.2636 req/s

APC opcode and APC Object caching:
PHP Generation Time: 0.03407 seconds
Number of includes: 81
Number of stat calls: 148
cURL time to start transfer: 0.03975 seconds
Apache Bench Concurrency: 260
Apache Bench Requests Per Second: 77.7214 req/s

APC opcode and W3TC APC Object caching:
PHP Generation Time: 0.03993 seconds
Number of includes: 102
Number of stat calls: 285
cURL time to start transfer: 0.04591 seconds
Apache Bench Concurrency: 200
Apache Bench Requests Per Second: 67.581 req/s

APC opcode and APC Object and Page caching with Batcache:
PHP Generation Time: N/A
Number of includes: Unable to collect
Number of stat calls: 41
cURL time to start transfer: 0.00316 seconds
Apache Bench Concurrency: 600
Apache Bench Requests Per Second: 147.2156 req/s

APC opcode and W3TC APC Object and Page caching:
PHP Generation Time: N/A
Number of includes: Unable to collect
Number of stat calls: 87
cURL time to start transfer: 0.00625 seconds
Apache Bench Concurrency: 500
Apache Bench Requests Per Second: 147.8425 req/s

Conclusions

I can state the following about just enabling APC in PHP, if you do nothing else, you should at least do this:

1. 170% PHP generation time improvement by enabling APC opcode caching
2. 172% Time to start transfer improvement by enabling APC opcode caching
3. 300% concurrency improvement by enabling APC opcode caching
4. 256% requests per second improvement by enabling APC opcode caching

I see performance improvements using both APC+Batcache and W3 Total Cache. However, in all tests, APC+Batcache seems to outperform W3 Total Cache, in PHP generation time, number of includes, number of filesystem stat() calls, time to start transfer, number of concurrent requests and requests per second with relation to concurrency.

I was able to push APC+Batcache to 700 concurrent requests, but req/s dropped. W3TC capped out at 500 concurrent requests, and would go no further, however 500 requests per second provided the highest req/s for W3TC.

W3TC does provide a lot of additional functionality to help reduce load on the server, such as tweaking client side caching, and using a CDN, where APC+Batcache does not, although there are small unitasking plugins that can add the missing functionality for you such as:

• WP Minify
• CDN Tools
• Use Google Libraries

APC+Batcache consists of adding 3 new files, and no new directories. The W3TC download consists of 60 new directories and 351 files. The directory listing level for W3TC being as deep as it is, 5 levels deep past the directory for the plugin itself, causes a significant increase in filesystem stat() commands.

Most shared hosting providers as well as many multiserver environments will often host their web roots on NFS, and the more filesystem stat() calls, the worse performance you will see, especially under higher load.

Something else to note, is a lot can be done on the server to also improve performance. You can also use caching applications that logically sit in front of the webserver to cache, instead of using caching plugins, which will also improve performance. There are probably eleventy billion ways to improve performance, so if in doubt, consult an expert to help.

I’ll try to make this comparison comprehensive, but it can be a little difficult to always cover everything.

The test environment:

• Toshiba T135-S1310
• Intel SU4100 64bit Dual-Core 1.3GHz
• 4GB DDR3 Memory
• Ubuntu 10.10 64bit
• Apache 2.2.16
• PHP 5.3.3
• PHP XCache 1.3.0
• PHP APC 3.1.3p1
• Memcached 1.4.5
• Pecl Memcached 3.0.4
• MySQL 5.1.49 No caching configured
• cURL 7.21.0
• WordPress 3.1-alpha (r16527) Default install with Twenty Ten and no plugins other than the one I mention below

The times are based off of the standard timer_stop() code often found in the footer.php of themes, in this case added using the wp_footer filter through a mu (must use) plugin:

<?php
add_action('wp_footer', 'print_queries', 1000);
function print_queries() {
?>
<!-- <?php echo get_num_queries(); ?> queries. <?php timer_stop(1); ?> seconds. -->
<?php
}

cURL was used to make the HTTP requests and grab the value from the comment created by the above code:

for (( c=1; c<=101; c++ )); do curl -s http://wordpress.trunk/ | grep '</body>' -B 1 | head -1 | awk -F"queries. " '{print $2}' | awk -F" seconds" '{print $1}'; done;

In each data set I gather 101 results and omit result 1 so that we only have results after the initial cache is generated. The tests are only performed on the home page.

The tests:

1. No Object or Opcode Cache
2. Memcached Object Cache with no Opcode Cache
3. Memcached Object Cache with APC Opcode Cache
4. Memcached Object Cache with XCache Opcode Cache
5. APC Object and Opcode Cache
6. APC Opcode Cache with no Object Cache
7. XCache Object and Opcode Cache
8. XCache Opcode Cache with no Object Cache

I didn’t evaluate eAccelerator due to the fact that it isn’t available in the Ubuntu repositories and I did not feel likely compiling…

The results (in seconds):

For a larger view of the spreadsheet above or if you cannot see it, take a look here.

These results are quite interesting and actually shocked me a little bit. The first thing that I found when developing an up to date XCache Object Cache plugin was that it can’t handle objects! So the plugin has to serialize all data when setting, and unserialize when retrieving. This of course is going to add overhead to every operation.

When I first tested the Memcached Object Cache I was surprised at how little it improved speed. It took me about an hour to realize that the comparison of just using Memcached was unfair as it didn’t include any Opcode caching, adding an Opcode cache brings it more in line with what I would expect.

Using an opcode cache improves performance by over 200% on a stock WordPress install without using any object caching. While APC and XCache provided similar results, my tests still show XCache to be ever so slightly faster as an opcode cache.

Where we see the biggest difference between the 3 of these caches when using APC for both opcode and object caching.

Assuming we are using both Opcode and Object caching here are the results from best to worst:

1. APC
2. Memcached (With either APC or XCache)
3. XCache

At this point the single largest failure of XCache is it’s inability to store objects, so I am pretty much planning on dropping XCache on my servers in favor of APC, which will be included with PHP as of PHP 6. I would likely still see marginal speed improvements using XCache on sites that I am not using XCache for an object cache, but on those that I am I’ll get much improved performance off of APC or Memcached.

Now why would I want to use APC over Memcached or vice versa? Well, the one thing that Memcached provides that APC doesn’t is the ability to share the cache between servers. In a load balanced multi web server environment, using APC you would be duplicating the cache on all of the servers as APC provides no way to share this data or allow for remote connections. Memcached however, being a PHP independent daemon can be used for pooling resources and allowing remote connections. You also can get more bang for your buck with Memcached in a load balanced multi server environment because of it’s pooling capability. The pooling capability allows you to dedicate say 128MB of RAM to each memcached instance and when pooled together will give you 128MB x N where N is the number of servers in the pool. Anyway, I digress…

In the end, if you have WordPress hosted on a single web server, APC is the way to go. If you are in a multi web server environment, Memcached is the way to go, but remember to install an Opcode cache as well. If you are crazy and just want to use more CPU cycles, XCache is the way to go.

Some of you may be thinking “why would I need an object cache in addition opcode caching, if the results are similar?” Well, under higher load an object cache will respond better than MySQL, even with MySQL caching. In addition, other factors with MySQL can come into play, such as connectivity to the MySQL server. It may be on another server, with not enough memory, slow disks, with an overloaded network, which decreases performance. Any time that an update query is run, MySQL will flush the whole cache. Another benefit, is we are rarely, if ever, going to use the data exactly as it is given to us from the MySQL query. In the end we are going to process the data before displaying, an object cache allows you to store the processed data, rather than the raw data from the query saving CPU cycles required for the processing. Individually these items may not consume much time, but added together and in a more efficient delivery system, this can make a huge difference.

Now for any of you who go run out and install Memcached, if you install version 1.4.x make sure you get at least pecl memcached 2.2.6 or 3.0.4. Memcached made a change that breaks deletes with earlier pecl memcached versions, which adversely affects WordPress.

A few additional things that I have been asked to talk about are using caching with a WordPress Network, output caching with Batcache and query counts. I promise to get to those, but I just wanted to get this out sooner rather than later.

Yo Dawg! We heard you like caching so we put a cache in your cache, so you can optimize while you optimize…Sorry couldn’t resist.

Many years ago, I don’t remember how many exactly, I was challenged by my High School math teacher, that if I could memorize all the digits of π that were on the poster (86) above the white board, that I would get an ‘A’ on the final without having to take it. I spent a week or two, I even wrote a program for the TI-83 to help me learn it. In the end I learned somewhere around 280 digits of π and got that ‘A’.

From then until now, I have always held a special place in my heart for this number of numbers. I own the movie π, I own the book a history of π and I have spent the last eleven to twelve years incredibly interested in this number. Back in 1998 or so I actually ran a site dedicated to π on geocities that is now long gone and forgotten.

So in keeping with the color scheme of that old site I used to run, which I now realize was very Matrix-esque, have a happy π day!

Happy π day!

PS: From my days as an astrophysics major, I was taught that really only the first 42 digits of π are useful for computational purposes. But what fun is that!? Keep learning and cranking out those digits of π!

Howdy, I’m Matt Martz. The majority of you probably know me as ‘sivel’ and I will be doing two separate talks at WordCamp NYC on Saturday, one in the Advanced Plug-in Dev Track and one in Beginners Plug-in Dev Track. The two topics I will be covering are Intermediate Plug-in Development Techniques and Writing Your First Core Patch. I will also be spending as much time as I can in the ‘Hacker Room’ helping people test and write patches for the upcoming WordPress 2.9. If time permits I’ll try do spend some time at the Genius Bar as well.

Giveaways

In both of my sessions I will be doing Twitter based giveaways. I will be giving away 5 items to randomly picked people who attend my sessions. If you want to find out what I am giving away you will have to come my sessions. Winners will be picked automatically at the end of each session using a WordPress plug-in I wrote specially for the occasion.

Intermediate Plug-in Development Techniques

We all strive to write good plug-ins. Plug-ins that not only function well, but plug-ins that have sexy code and use the WordPress APIs whenever possible for tight integration into core. I’ll go over some of the techniques which I believe will help take your plug-ins to that next level. Techniques will include:

• Splitting Plug-ins into Multiple Files
• Tips on When to Load
• Using Classes
• Localization
• Ajax
• If time permits I will try to touch base on a few other items

Writing Your First Core Patch

There are a lot of people out there that I see every day saying they found a bug or want a feature, but in the end never do anything about it. I’ll go over testing to verify the bug, getting assistance from the community, using the WordPress provided resources, explaining Trac and ticket fields and if time permits giving a few demos for actually creating that patch.

To make the service even better you can integrate it into your web browsers search bar for even faster lookups.

I have taken his service one step further and written an IRC bot that is currelty sitting in the WordPress IRC Channel. To use this bot type something in the form of .codex get_pages and it will query WPLookup and return the resulting URL back to the channel.

See the following links for his release announcements about this new service:

1. Find WordPress Function and Template Tag Documentation – Fast
2. New WPLookup Features – Set WPLookup as a browser search engine

The largest amount of time it took to get this finished was recreating the rewrite rules for Zenphoto. The rewrite rules can be found here. I haven’t tested all of the rewrite rules yet, but from quickly browsing through my gallery everything appears to be working well. Keep in mind that these rewrite rules were designed for a site where Zenphoto lives in the root. If it lives in a subfolder the rewrite rules will need to be modified accordingly. I can probably help if you ask nicely.

I have also moved over numerous WordPress sites to nginx. I won’t post their rewrite configurations here because you can find them easily using Google.

You can find the nginx rpms for EL4/EL5 in the Fedora Project’s EPEL repo.

If you have any nginx related configuration questions feel free to ask, I’m getting pretty familiar with the app.

I wanted to verify that XCache was working, so I copied the XCache admin directory into one of the document roots and tried to login. To my surprise I was not able to login even though I was using a known good md5 password hash in the XCache configuration.

It took me about a month of searching to find out that the server variables PHP_AUTH_USER and PHP_AUTH_PW don’t seem to work with mod_fcgid. This Lighttpd forum entry led me to the XCache admin config.php.

Here is what is required:

1. Create a file in your XCache admin directory called config.php
2. In this file add the following, using the information you provided in the XCache configuration:

   <?php
   $_SERVER['PHP_AUTH_USER'] = 'admin_username';
   $_SERVER['PHP_AUTH_PW'] = 'admin_password';
   ?>
   

3. Configure some other means of authentication such as htpasswd or htdigest