Backing Up Lots of WordPress Clients: How to Make Life Easier

The Task

Recently at work we were about to be handed the task of making sure that all of the client sites were backed up. For a while, some other people we handling that task, but it was being handed to the developers since we are the ones that actually maintain the websites. There are about 30 sites we are managing so that is a lot to backup.

Some Considerations

Before we could really tackle the task of handling the backups we had to make a lot of considerations. The first was understanding exactly what we needed to backup. Next we had to find the best method for doing the backups themselves because it does take a lot of time. We also needed to know what sort of controls and monitoring we needed to have over these jobs.

Why is this thing so slow?

The internet at the office isn’t the greatest. It is an old T1 connection that isn’t the fastest but it gets the job done.  Originally the jobs were being performed by using FTP and downloading all the files. That is obviously a problem for several reasons. Doing that many FTP downloads takes a lot of time on our bandwidth, secondly unless you automate the backup, it will take time to go to each site and download the files. There is also the issue of what happens if the power goes out, or the internet goes down that night. It wouldn’t be unheard of to have that many connections happening for a length of time taking down the network temporarily, even for a few seconds.

The Options We Looked At

There were several options we were considering when we talked about this process. First we considered writing a script for each site to do the backups and using a cron to run them. While that is probably the best idea especially from a power user standpoint, I’m the only one there with the level of backend server administration experience. My concern was that others wouldn’t be able to fix an issue if I wasn’t around (bus plan thinking).  Next we discussed doing the backups internally via FTP or a backup program, but as I mentioned before there were far too many issues facing that type of backup. Not only that, keeping backup copies locally really doesn’t help you in the event of a fire unless they are backed up to a medium and stored in a fire safe.

The final option we looked at was using a plugin since the sites we build are on the WordPress platform. Using a plugin would make it easy for anyone to configure as well as troubleshoot. We looked into several but finally decided on using WPBackup Pro. While it isn’t the cheapest plugin, it certainly has a lot of worthwhile features.

What We Liked About WPBackup Pro

One of the things we really liked about the plugin was that it allows you to backup to cloud storage providers like Dropbox and Google Drive. We use Google for emails and thus we have storage with them as well.  What makes it so perfect is that it centralizes the backups in a place where we have control over access, and we know that it is relatively secure, plus far more redundant than any local server we have. Scheduling the backups is really easy to do as well, you can literally run a Job Wizard and backkup in minutes. The Google Drive setup does take a little of figuring out just because the API requires a few extra steps to connect, but once you do it one time, each new time goes faster and faster.

Our Setup

The setup we followed uses these principals:

  • Files should be kept for 90 days. This gives us three months of files. In the event of an unknown malware infection it gives us enough of a time window to hopefully restore from a version before the infection.
  • Databases should be backed up nightly. This helps in situations where data  becomes corrupted.  Since the database gets updated a lot, it makes sense to back it up a lot.
  • Site files should be backed up weekly. They don’t change very often so there isn’t a point in doing it frequently.
  • All plugins, and upper level directories should be backed up. Sometimes there are directories above the WordPress install that we use, so we should back those up.
  • Alerts should be sent whether the job succeeds or not. This is something I learned in IT. If you are only getting alerts if the job fails, you have no way to know if the job is stuck unless you look. While that might be easy in a centralized backup application, you don’t have that with 30 client websites. This allows us to count the emails really quickly and see if we have our 30 successes from the previous day.
  • TarGZ should be used as much as possible to limit filesize.
  • Backups should be staggered to limit server load. Each job is given a 5 minute window (unless it needs longer) so that there are never two jobs overlapping.

All in all after setting this up it seems to be working really well. There were a few hiccups along the way where a job or two didn’t run, but after resetting the job it seemed to go away. This was definitely a great investment because of the amount of time it saves as well as how easy it is for anyone to go in a do a backup.  I’m glad that we didn’t go the FTP commando, or scripted way. If you manage a lot of sites I highly recommend checking the WPBackup Pro plugin out.


I Have Absolutely No Idea What I am Doing – A Guide to Learning New Things

People think I’m some sort of expert

Everyone in my family thinks I’m some sort of computer expert. I’ll be honest and state that I know very little compared to a lot of people out there in the world.  The reality is that the skill I do possess is being able to look up information and process it in a way that allows me to complete whatever it is I am working on. Recently I started working on learning IOS development and while I understand very little of objective-c, I am still able to get things working by understanding the problem I have.

One of the best pieces of advice I was ever given was that “It isn’t the solution that matters, but the question that you need to ask”. When you know what question to ask then you know how to find the right answer to solve your problem.

Don’t worry if you don’t understand something 100%

There are plenty of times that I have worked on a project, even if I didn’t understand every single bit of what a function did or how some class was being used. In the end as long as it works that is all that matters. Now I know that some people are going to groan and say that something may not be efficient or possibly insecure, and you’re right those things do matter, however in the grand scheme of things if you are just starting out learning something efficiency doesn’t matter nearly as much as being able to put together a working project vs giving up.

Know where to look for what you need

When I started my new job as a developer I struggled a bit at first because most of my experience was working on the backend. People think that front-end dev is easier because you don’t have to do as much heavy lifting yet they don’t consider how hard things like UX and design can be. Sometimes the designers aren’t working with you to understand the client needs and expectations of the project and it is up to the person in dev to help make them aware of that.

That is when I realized my issue was that I wasn’t really following any blogs that talked about design, ux, wordpress development, or anything else for that matter. I spent a few days looking up blogs that I should be following and in not time I was catching up to current trends and techniques. It really can make a difference if you take the time to look up and follow articles on topics that matter to you. My feed currently has about 20 sites that I visit on a regular basis and it has helped me so much.

Be the developer you want to be

I think that everyone should strive to be the best person that they can be in their field. I’ve worked in IT, tech sales, and full stack development and I always push myself to be the best that I can be. When I’m at home I’m still learning new things and keeping current on the world of development. There are always at least three or four books on my tablet at a time that I am reading through and I”ll always try to pull it out at least once a day to flip through a few pages.

Don’t get discouraged by not knowing something, often times it is your drive to learn that will help pull you through. There have been many times in previous jobs where at the beginning  I was looking at something that I couldn’t understand and was thinking “How am I going to pull this off”. When you take the time to break your problem into smaller pieces, and understand how every piece works with one another, you will know the right questions to ask, and you will find the answers you need.

Keep on dev’ing on!

 

 


My Workstation Setup

I recently put a little bit of time into my computer, which I have been neglecting for a while. When your career is based around using a computer it makes sense to have a nice setup that is easy to use and incorporates great software. Plus you have to have a couple nerdy desk ornaments that you can play with!

The Hardware

My main computer is an Intel I7 based Windows pc with 16GB ram and 2TB of randomly assorted hard drives, one solid state, one velociraptor, and the rest all standard platter drives. My video card is an old GTX 560 which should probably be upgraded. As always, I am using an Asus motherboard. The case is a Corsair Obsidian 800D. The secondary computer is a Mac Mini that I shamelessly took from my fiance. 😛

How Many Monitors?

Currently I have two Asus VE248H monitors and an old Hannspree HF259h that I got at Best Buy when I used to work there, which was like 7 years ago( wowzers). I actually have several monitors that I still need to set up for various smaller devices including a laptop with Kali Linux and my Raspberry Pi.  The two Asus monitors are connected to my Windows PC and the Hannspree is connected to the Mac Mini.

More monitors means more screens to look at more things at the same time.

Work Environment

Since I do a wide range of computer magic, I like having as many options at my disposal as possible. That is why I have a Mac, Windows, and Linux computer all ready to go. Currently I am using Synergy to work between the Mac and Windows PC using only one keyboard and mouse. If you haven’t used that software, you absolutely should check it out. I realized in the end that I didn’t need three monitors on one computer, especially if I was going to have multiple computers.

Even though at work I probably wont be able to get away with using it (yet) Sublime Text 2 is the IDE that I am growing to love.  You absolutely cannot beat all of the keyboard commands and functionality that program has.

I’ve finally started to use version control as well.  Thanks to my buddy Issa, I can now fumble my way through making commits and keeping track of the terrible mess that is WordPress development. The GitHub desktop app failed me on Windows so I’m just using the command line which isn’t bad at all.

Now that I am using a VCS the next issue would obviously be deployment. There is where my trusty buddy Beanstalk comes in. It acts as repository to store all of my version control, but it also has the power to do deployments.  So now instead of having to use git only to go FTP commando I can now just push the files, and then deploy to the site remotely.  There’s even a phone app! All in all I am very impressed with it.

Another aspect of my work environment I’ve started working on is developing locally.  For the longest time I was guilty of using the FTP feature in most of the popular IDE programs (I’m looking at you UEStudio/DW). At first I couldn’t see the point in working that way, it seemed like a hassle. The reality is that when you are working on a website you should never EVER be working on the files directly in a production environment, especially if it is something that can impact the user experience or potentially break the site.

That is where XAMPP comes in. It makes setting up my computer as a web-server extremely easy, clicking a few buttons easy. I really wish I could use MAMP but sadly they only have a beta on windows, which works great but only lets you work on one site at the moment.


A Post A Week

What I want to do

I don’t use this website as nearly as much as I should. I have decided that I want to start posting at least once a week, and then hopefully trying a blog a day challenge for the month of June.  I’m sure this is probably going to be harder than what I think but whatever, I’ll find a way to make it happen.

How I plan to do it

My plan is to focus each week on a different topic.  I’ll probably start off with a post about some cool new technology, then post about something I am working on. Maybe after those two posts I’ll try my hand at writing a How-To. For the final post I will talk about all the challenges I faced in doing this.

Isn’t one a week sort of…lame?

Well yes it is actually! However, I really do need to start writing more and if I have learned anything it is that you have to set realistic goals and with as busy as I have been, this is probably the most real I can get.