Part 1: The problem

I decided I wanted to migrate my old Land of Devastation website at http://www.landofdev.com/ over to a subsection of http://www.smbaker.com/. There are several reasons for this, chief among them being that the Land of Devastation website doesn’t really stand by itself and doesn’t have a whole lot of pages. I’d rather discuss all of my door games together at my main site.

So, as part of the process I setup my provider to map the domain name www.landofdev.com into my existing www.smbaker.com website at a subdirectory of /landofdev. All I needed to do then was some simple rewriting or redirecting of pages that well into /landofdev/ to send them to the appropriate location in the main website.

Well, I thought it would be simple…

Part 2: Understanding mod_rewrite and wordpress

I spent a great deal of time one night trying to figure out how the wordpress mod_rewrite rules worked, and more specifically trying to figure out how to internally rewrite pages from one of my domains into another. Let’s start by looking at the rules that wordpress puts into the .htaccess file:

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>

If you’re familiar with mod_rewrite then this probably makes a lot of sense to you. If you’re not familiar with mod_rewrite, then let me explain it. The first RewriteCond checks to see if the current request maps to an existing file. If it does, then the remainder of the rule is skipped. The second RewriteCond checks to see if the current request maps to an existing directory. If it does then the remainder of the rule is skipped. These two checks allow you to mix static webpages on your site with Wordpress permalinks. For example, say my website has a file:

/foo/bar.html

If a user request arrives for /foo, then the ‘-d’ rule sees this and aborts any further rewriting. If a request comes in for /foo/bar.html then the ‘-f’ rule sees this and aborts further rewriting. On the other hand if a request arrives for “/foobar/some-permalink”, then this request will be rewritten since /foobar/some-permalink is neither an existing file nor an existing directory.

Finally, the last line of the rule is the RewriteRule. It says to that anything that has made it this far should be rewritten to /index.php. Index.php is the main entrypoint for your wordpress site. The result is that anything that didn’t exist as a file or a directory gets sent to inedx.php where wordpress can do with it whatever it wants (normally, interpreting the URL as a permalink).

Part 3 – What I wanted to do and why it doesn’t work

Late one night I decided I wanted requests on my site for /landofdev/ to be rewritten to /games. “games” was a permalink on the website. This sounds like a pretty simple rewrite rule, doesn’t it? You might think of coding it up like:

RewriteRule ^/landofdev/$ /games [L]

Unfortunately, it doesn’t work. Not worth a damn. The RewriteRule itself goes off as expected, rewriting a request for landofdev/ to /games. Then it hits the wordpress rewrite rules, which further rewrite /games as /index.php. So far, so good – we’ve managed to route our request into wordpress. However, wordpress doesn’t really care what the page was rewritten to. It uses the REQUEST_URI variable from the web server when parsing the permalink. Mod_rewrite doesn’t modify the REQUEST_URI. So, although my little rewrite rule managed to get /landofdev/ routed to the right place, wordpress never saw the “/games” that I’d intended to rewrite it to, instead getting “/” out of REQUEST_URI, trying to find a permalink named “landofdev” and failing with a 404 error.

minor confusing footnote – the reason REQUEST_URI for http://www.landofdev.com/ contained in “/” instead of “/landofdev/” is because of whatever mojo the web provider is doing to map that domain into that subdirectory.

Part 4: How I bent wordpress to my will

I eventually solved with problem with an ugly hack. I placed a wordpress index.php inside of /landofdev/ on the website and added an ugly little if statement to it.

<?php
define('WP_USE_THEMES', true);
if ($_SERVER[REQUEST_URI] == “/”)  {
    $_SERVER[REQUEST_URI] = “/games”
}
require('../wordpress/wp-blog-header.php');
?>

Additional if statements could be added to remap additional pages, and you can also use wp_rewrite to help out a bit (one if statement to do a blanket redirect of everything in /landofdev, plus wp_rewrite regex rules to send individual pages to the right spots)

One small point of clarification – you might have expected me to use “/landofdev” in the if statement rather than “/”. On the website, “/landofdev” is actually representing an alternate domain name. I’m not sure exactly how godaddy does this, whether it’s a virtual host or some other magic, but the REQUEST_URI that show up at /landofdev are relative to the directory. Thus if someone requests http://www.landofdev.com/” it arrives in the /landofdev directory with a REQUEST_URI of “/”.

Finally, if someone does have a *better* way to do this, I’m looking for suggestions.

Well, there’s a very simple fix for that, although it’ll require a little bit of modifications to the header.php files of your themes. Enter this code into the header.php file of the theme that you’re using. A safe place to put it is right before the </head> tag.

<?php
  if ($custom_css = get_post_meta($post->ID, 'custom_css', true)) {
    echo '<style type="text/css">' . $custom_css . '</style>';
  }
?>

Once you’ve done this, all you need to do is to create a custom field called ‘custom_css’ on your blog posts and put your custom CSS in there. It ought to look something like this:

The ‘Value’ text entry box is small and somewhat inconvenient to type in, but you still fit an awful lot of CSS in there. For example, for the Rollover Button tutorial I needed to include about a dozen lines of custom css.

One caveat is that it’s unclear exactly what will happen if you have multiple blog posts displayed on the same page and you’ve created conflicting CSS for them. I’d suggest being careful and keep ponential conflicts in mind, or do what I do and operate your blog in one-post-per-page mode.

Let’s start by making some button images. We’ll need two of them, one for the button’s normal state and the other for the mouseover state. I used a simple paint program to create these two buttons.

This will be our “up” state button. It’s a little darker and has normal text.

This one will be our “down” (or mouseover) button. It has bold text and a lighter color.

Our first step is to combine these images. Individually they are each 80×32 pixels in size. We’ll create a new image that is 160×32 pixels and put the up image on the left half and the down image on the right half. The combined image looks like this:

Let’s start by creating the HTML where our link will go. Rather than using a tradition <img> tag, we’re going to use a <span> tag. Here is the html:

<a class=”smbakerbutton” href=”http://www.smbaker.com”><span>smbaker!</span></a>

As usual, the hyperlink is made with an <a> tag. That should look very normal to everyone. The button itself is going to be displayed in the <span> tag – that’s where things get weird. Where’s the button, you say? Well it goes inside some CSS that we’re going to write.

.smbakerbutton {
    display: block;
    width: 80px;
    height: 32px;
    background: url(smbakerbutton_rollover.jpg) no-repeat 0 0;
}

Let’s examing this code in detail. The display: block tells the browser to render this as a block-level element. ‘width’ and ‘height’ are the height of our button (which is one half the width of the double-wide image that we created). The ‘background’ statement says to use our image as a background image. So what this has done is to turn our hyperlink into a 80×32 rectangle with our button image as the background.

Next we need to handle the mouseover state:

.smbakerbutton:hover {
    background-position: -80px 0;
}

So what we’ve down is every time the mouse is over the button, we shift the background image 80 pixels…. Well since our image is 160 pixels wide, and the other half of the image is the “down” state of the button, that’s going to cause our button to display the “down” state.

Finally, we don’t want the text in the span tag to be displayed, so we’ll turn off the span’s text output:

.smbakerbutton span {
    display: none
}

Here is what our button looks like. You can mouseover it and it should change state:

smbaker!

Here is all of the CSS in one big block:

.smbakerbutton {
    display: block;
    width: 80px;
    height: 32px;
    background: url(smbakerbutton_rollover.jpg) no-repeat 0 0;
}
.smbakerbutton:hover {
    background-position: -80px 0;
}
.smbakerbutton span {
    display: none
}

Remember that the CSS needs to go in your stylesheet, or needs to otherwise end up in the header of the page. Putting the CSS in the body of a wordpress post won’t work. I’ll tell you a way to get around that in the next blog entry!

Something that a lot of people may not know is that you can bypass DNS entirely on your local computer by editing your /etc/hosts file. The /etc/hosts file contains static linkings between hostname and IP address. It’s been used in the Linux world for a long time. However, windows puts it in a strange place. In Windows Vista and Windows 7, the file name is:

c:\Windows\System32\drivers\etc\hosts

You’re also going to need administrate privileges to edit the file. The easiest way to do it is to simply use notepad. Search for notepad using the windows start button, but rather than left-clicking on notepad, right click and select “Run As Administrator“. Windows will do the usual user access control screen flash and prompt you to ok administrator privileges. Once that’s done and notepad is on screen, simply use File:Open and enter the pathname c:\Windows\System32\drivers\etc\hosts.

Entries in the file are pretty simple. They’re just an IP address followed by some whitespace and a list of hostnames. For example, if I wanted to link the IP 123.456.789.012 to both www.mysite.com and mysite.com, I could enter the following:

123.456.789.012 www.mysite.com mysite.com

Make sure not to mess with the entries for 127.0.0.1, ::1, or localhost. Those are special entries that programs on your computer can use to identify your computer itself.

Remember that once your domain names changes actually take affect in the DNS system to remove the manual settings from the hosts file. If you leave them in permanently, then you would be masking potential DNS failures as well as setting yourself up for a great deal of confusion in the future if your provider ever changes your IP address.

I found this technique pretty handy for migrating new domains over to my shared hosting account. I could create the new site on the new provider, migrate all of the files and get the site setup using the changes that I wanted, and then do the little hosts trick to preview the site as it would appear to users. Once everything was to my satisfaction, then I contacted the domain registrar to move the actual DNS records.

One potential gotcha — I found that occasionally it would take a while for applications on the computer to learn of the changes to the hosts file. This was particularly bad with Internet explorer. If you have a web browser open and looking at the hostname before you change, you might need to exit the browser and restart it for the changes to take effect. In the worst case, rebooting your computer would solve the issue.

Experiment conditions – A random URL fetched every 5 minutes from a pool of 27 URLs. Only the HTML is fetched, no images or other files. Web hosting using godaddy unlimited linux plan. Database with direct external access enabled. WP-Super-Cache enabled in half cache mode.

The numbers were chosen particularly to cause a lot of cache misses (27 pages with a 5 minute interval is greater than the cache expiration time). Approximately 32% of the page retrievals resulted in cache hits resulting in less than 1 second retrieval time. A total of 71% of the retrievals occurred in 2 seconds or less, and 85% in 3 seconds or less. Those are the retrievals that I’d consider acceptable. However, 7% of the requests took greater than 10 seconds. That’s getting into the range where people will think a site is down or unresponsive. 4% of them took 20 seconds or more. Two of the requests actually failed due to database timeout.

One of the options when I created this database was whether or not to enable external direct database access. I originally chose this option so that I could do easy database backups via mysql to my linux boxes at home. According to the godaddy website, databases with external access are hosted on a different server than those without. For this reason, I decided to re-import the wordpress site into a database without external direct access and see if there was a difference in performance.

Experimental conditions – same as above, but database with external direct access disabled.

This time 97% of the requests were 2 seconds or less, and 100% were 3 seconds or less. There were no page retrievals over 3 seconds. There were no timeouts.

This is good news as these are finally numbers that I’m confident will lead to a successful website using wordpress. I’m not sure whether the issue was that the databases that allow external direct access are slower than those that do not, or simply that I’ve been lucky enough that I migrated to a database server that receives less load than the others. Pinging the two servers (the original one that allowed external access and the new one that does not) reveals that they are as expected two different IP addresses, and likely two different servers.

I’ll keep monitoring the site with a long-term benchmark to see whether the new performance continues to be acceptable.

Installation was fairly easy from Microsoft’s site with a few caveats. The installer automatically tries to install a bunch of Microsoft bloatware that I didn’t want (Mail, etc). It also wants to set my default search provider to Bing (YUCK!) and set my default homepage to MSN (even bigger YUCK!). Any application software that dares attempt to touch my search or homepage settings automatically deserves placement in the ‘hall of shame’. So, –1 points for trying to muck with settings that it shouldn’t muck with.

Okay, so now we got the installer installing Windows Live and only Windows Live. That’s good news. It automatically prompts me for blog settings and I enter the info for my site. It detects that XMLRPC is disabled on the site, so we have a quick interlude to login and enable it. It now tries to download the theme for my site and fails with an undecipherable error message (something like “Windows live failed to download the theme from your website”). Okay, that’s useful. Well, previewing the theme is not very important. My brain is good enough to render the page in living color even if Windows Live Writer is not.

The next thing is does is pop up a page to enter a new post. That’s where I am right now, typing into the post. There’s a dropdown for setting categories, easy enough. There’s a text box for settings tags. Up at the top is a <Publish> button. I’m going to click that and see what happens.

The publish button popped up a quick dialog that showed the post being uploaded and then claimed it would show me the new post of my site. Well…. it tried but my site’s home page is set to a static page instead of the blog page. Can’t really fault the software for this unexpected yet minor inconvenience. Click my blog link, and there it is…. This page is exactly where it’s supposed to be, looking like it should. Now lets hit <Publish> again and see if it updates it.

Yep, update works as expected.

How about I put in a hyperlink. It’s time for a table with a couple of images:

Interestingly enough, when I typed in some URLs for the above pictures, it seems to have downloaded them and thinks it’s going to re-upload them. Well, I can see where that would be useful when creating posts, but I think it would also be quite handy to be able to enter a link to an existing image.

Problems (just a laundry list of things that I can’t seem to get to work properly) –

• I can’t figure out how to set the page template when creating a new page. My site uses a variety of custom templates for different classes of pages.
• Copying a formatted page out of my web browser and pasting it into Windows Live Writer didn’t preserve the html tables. Tables are preserved when using the wordpress built-in editor, so copying the page into wordpress and then editing it later with Windows Live Writer is an option.

Generally I find the editor pleasant and easy to use. It’s certainly better than using the built-in wordpress editor.

As we can see, the horrid performance that we were experiencing without caching is now gone. 98% of the requests occur in 0.2 seconds, which is the same time as fetching a static page from a static (non-wordpress) website. 2% of the requests take about 1.3 seconds and those are likely the cache misses. There were none of the aberrant 10-second or 20-second spikes that were so pesky when operating directly on the database. WP-Super-Cache is clearly a win when it comes to masking bad database performance.

This has eliminated the primary concern that I had with migrating to wordpress/godaddy — page generation times in the 10’s of seconds are not acceptable.

The remaining concern is ease of authoring. Authoring and in general mucking-around with wordpress administration inherently must use the the database. This means that it can’t take much advantage of WP-Super-Cache/WP-Cache. If the website becomes too onerous to maintain, then I’d still have to abandon the idea.  As just about everything is becoming dynamic these days, this hilights the importance of looking past simple static page fetching speed and evaluating your provider’s database performance. As time goes by, websites are going to use more database, not less.

Updated — I did, for the most part, resolve this issue to my satisfaction by changing some database parameters and adding a cache mod to Wordpress. Full details, with updated benchmarks using the new configuration can be found on this page: http://www.smbaker.com/more-wordpress-benchmarks-on-godaddy-hosting

However, no sooner did I get to doing some serious site work and I experienced a <gasp!> timeout from the website. I haven’t had a timeout on one of my websites that I can remember in years. So I decided to go to the root page of the wordpress blog and hit the refresh button. Sure enough, there was a bunch of variability in the page generation time.

Being a network guy, I constructed some simple benchmarks and set them to work for about a two hour period while I was off to lunch. Let’s cut to the chase, the first histogram is below.

So what we have a bove is a histogram that shows page retrieval time on the x-axis and number of pages on the y axis. I repeatedly fetched the wordpres homepage (http://www.smbaker.com/wordpress/) of this website every 15 seconds and ran this test for about two hours.

Experimental conditions – the website is a simple wordpress CMS configuration, about a dozen pages online at the time of the benchmark. The WP-Super-Cache/WP-Cache addons are not enabled. The website is receiving no other traffic than what is being generated by the benchmark. The test was run from 11:00am to 1:00pm pacific time. Web package is the godaddy ‘unlimited plan’.

The first thing we notice is that 72% of the pages were served in 0.6 seconds or less. 84% of the pages were served in 1 second or less. Unfortunately, that’s where the good news ends. 8% of the pages took over 2.5 seconds or more to fetch. IMO, once we start getting to more than a couple seconds that’s where websites start looking sluggish. So, 8% of the people who visit this very simple wordpress site will experience a sluggish site. Let’s look into the outlying data a bit closer.

4% of the pages took more than 5 seconds. Of that, 2% of the pages actually took more than 10 seconds. 10 seconds???? A couple of page retrievals took in excess of 20 seconds. This is the year 2010, not the year 1995.

The first thing to do is to factor out non-server related factors. Things like my PC here at home, my ISP, the network connections between my ISP and godaddy, routers, dropped packets, etc. All of the stuff that makes the Internet unpredictable. I was thinking of this ahead of time and I ran a benchmark fetching a static non-wordpress page at the same time as the above benchmark. The “static” page has no PHP. It doesn’t use any database access. Here’s a histogram of its’ behavior:

Here we can see that on a non-dynamic/non-database page, all of the accesses occur in 0.2 seconds or less. That’s enough to convince me that the issue isn’t my ISP, my home computer, or any router in the pipe.

As further anecdotal evidence while trying to update this blog post, I experienced three incidents of timeouts and “page unavailable” errors when trying to commit changes to this draft. I’ve also experienced six more incidents of extremely long load times (greater than 20 seconds) while trying to open simple wordpress interface pages (the edit page, the categories page, etc). I suspect that the database performance is even worse at storing information than retrieving it.

Updated — Read the follow-up entry: http://www.smbaker.com/more-wordpress-benchmarks-on-godaddy-hosting