Apologies

Posted on

Some of you may have noticed that our site was down a good portion of yesterday. My apologies. In preparation for our trip, we have had come up with a way to do our blogging and journaling offline, since we will not likely have consistent internet access in Europe. In order to get the correct functionality, we had to switch web hosts, which had a few hiccups.
For those who are tech-oriented and interested in our planned process, it is outlined as follows. For those who are not, you can be done reading now.
Since we will be hauling my laptop all over Europe, we have set up a local Apache2 server that runs on Ubuntu. On this server, we have a virtual host pointing to www.thecouplethatdoesthings.com.  Here we have our site duplicated – along with the mysql backend. Of course there are a few wordpress configuration files that needed to be changed, but these are few and fairly trivial.
In the field, we will use a 7.5″ EeePc that we picked up off of KSL for under $100.  I built a crossover ethernet cable that we will connect the EeePc to my laptop with in order to create a router-less and low power network.  By setting Bree’s hosts file to point to the server’s static IP, she can then point a web browser to www.thecouplethatdoesthings.com. We can then blog away, while simultaneously reviewing, processing and backing up our photos.
Once we do have an internet connection, we will push all of the changes made up to our hosted website. Since I keep our website’s history through a git repository, our original method involved scanning the logs since our last push, collecting the files, and ftp’ing them up to the server. This worked, but wasn’t really that fast. Additionally, we pushed entire files, instead of just the deltas. This is where rsync comes in. Rsync uploads the deltas which conserves bandwidth, uses compression, and is really quite snappy.  We of course have to tell rsync to ignore our syncing scripts, some of the cm configuration files, and the git repo. In order to use rsync, though, we had to switch our webhost to hostgator, which includes shell access without charging an exorbitant fee, as did our last host. This is why our site went down.
Since our site also uses a mysql database backend, we have to also push that to our hosted site. Since our database is not yet very large (a couple of megs at most), we have taken the brute force approach. I merely perform a mysqldump, and then use sed to replace all urls with those corresponding to our live site. Our new web host allows remote mysql connections, so I can then connect directly to the remote database and source the dump. In the future, if our database becomes very large, or it takes too much time to upload it, I will have to figure out how to use mysql’s binlogs to replace only the changes. As of right now, though, that doesn’t seem entirely necessary.
While this process may sound complicated, it really isn’t too bad. The whole process has been automated by a handful of bash scripts, so a sync for one or two posts takes on the order of 5 or 10 seconds – where most of that is spent establishing the remote mysql connection. I can live with that 🙂

Addendum:

After encountering a couple of problems with the brute force approach to pushing our mysql database to the server, I have opted to use binlogs for the database, so that we only push changes.  This is actually really cool.  The database logs all events that are not read-only to a file.  Those logs are then pulled, the logs purged, and the commands applied to our remote server.

Oh, and one more thing – a big thanks to Tyler from www.goingslowly.com.  Our process was heavily inspired by Tyler, who has what is probably the best cycle touring blogs out there.

Categories

Leave a Reply