Cloud Backup | ASchroder.com

Posted on October 25, 2011

Magento Cloud Backup 0.3 RC – new features and an end to the beta program in sight

We now have over 300 downloads of our Magento Cloud Backup extension, with users storing over 300GB of their store data safe offsite using Amazon S3. As we near a post-beta release I thought it might be worth announcing an early RC, for both initial feedback from other developers/admins and because some of the features in this RC have been asked for, and I’d like to allow those users to gain early access to features they need.

What’s new in CloudBackup?

Auto-cleanup of old backups
Probably the single biggest feature request is the ability to delete old backups automatically. I have implemented a simple, delete backups older than 3 months process. However if you want more or less backup storage, it’s easy to change like so:

Edit Backup.php and change private $NUM_MONTHS = 3; on line 31 to your desired number of months.

(I’ll probably make this a fully fledged configuration option for the actual RC when it goes live on Magento Connect.)

You need to enable the setting as shown in the screenshot above. Once enabled a nightly cron job will look for backups that are older than the specified threshold, and delete them from S3.

Native Database backup support

This is a MySQL only, power-user only feature that allows you to skip the Core backup if it takes too long on large stores (>2GB of database data).

It’s important to note a couple of things with regard to running native Database backups. The first is that while it’s running the mysqldump will take a lock over the database, for really really big databases that could be a problem as it may block checkouts/new accounts, for example.

The second is that mysqldump is run via a system() call this means you need security setting on PHP to allow this. It also means the mysqldump binary needs to be accessible to the PHP webserver user. To help that I have added a simple variable in Native.php (the native equivalent to Db.php provided in Magento core).

I’ll just let the code/comments do the talking with regard to configuration:

// if mysqldump binary is not on the webserver user's path
private $PATH_TO_MYSQLDUMP = ""; 
 
// For example on a MAMP based OSX install you might use: 
// private $PATH_TO_MYSQLDUMP = "/Applications/MAMP/Library/bin/";

This feature should really only be used by people who see it and think ‘great the answer to my prayers’ anybody else is likely to get themselves in to difficulty – it requires fairly deep access to your server environment – For example; if you’re on GoDaddy hosting, I’d be very surprised if you could run a native backup from PHP!

Trying the new features

You can get early access to the code over at Github, we’re using Github pages, it’s really quite a nice feature.

Once you have the code, you can install it manually in whatever manner you like, here’s some suggestions I wrote up a while back.

If you’re patient, I’ll have the RC release up on Magento Connect within the next few weeks. It needs a little more polishing before I’ll be happy with it though.

The End of the Beta Program

We’ve been running the beta program for over 12 months and I consider it a great success from a technical point of view – the software has matured gradually and with plenty of testers I feel comfortable about the quality of the backups being created, and the consistency with which they’re being made. It probably doesn’t take much business savvy though, to realize that with 300 downloads, and a bit over 300GB of data in a first-GB-of-data-free beta program, we’ve been shouldering quite a burden for everyone’s backup storage.

As I’ve said before, we would never take anything away from those that have been helping us test, so I’d assure everyone upfront that the initial GB of free storage will remain for those already using the extension. What will most likely happen once the extension is released as a 1.0 stable, is the extension will cost some small amount to buy, or the first GB won’t be completely subsidized. I’ll be sure to update everyone once we are clear on which direction the extension will go.

Anyone who has had a beer with me and endured my rant about the folly of selling not-scarce digital copies of things (including software) will know where I stand philosophically on the matter.

In any case, if you are considering offsite backups for your Magento store, or your client’s – now would be a pretty good time to jump on the bandwagon.

So that’s pretty much the update on our Magento Cloud Backup extension, if you have any questions/gripes, please don’t hesitate to comment below!

Posted on April 4, 2011

Cloud Backup Case study: Detecting if Magento cron is running in your extensions

We released the latest version of our Magento Cloud Backup extension, for sending a full site and database backup offsite to Amazon’s cloud storage service S3. You can read about the latest release over on the World Wide Access site. In this post I wanted to quickly cover the method I used for detecting if cron is running on a Magento store. This was one of the new features added to help new users identify possible problems.

I think it’s important for many extensions that rely on cron to check if it is running, because otherwise it’s possible for users to have a false sense of security (particularly around automated backups). This technique I mentioned briefly back in October at the Magento Developers Paradise. This more fully worked example will hopefully be useful to others.

The logic is this: If we have no pending jobs, or no successful jobs in the Magento cron table, then cron is probably not running (either it never has, or only has once).

$schedules_pending = Mage::getModel('cron/schedule')->getCollection()
        ->addFieldToFilter('status', Mage_Cron_Model_Schedule::STATUS_PENDING)
        ->load();
$schedules_complete = Mage::getModel('cron/schedule')->getCollection()
        ->addFieldToFilter('status', Mage_Cron_Model_Schedule::STATUS_SUCCESS)
        ->load();
 
 if (sizeof($schedules_pending) == 0 || 
        sizeof($schedules_complete) == 0) {
                // cron probbaly isn't running
}

With this code run from within a frontend model, or a controller, you can effectively inform your users that cron is probably not running. It would be quite a nice service actually that just automatically calls cron for Magento stores via the web.

It would also be possible to look at when the last successful job completed, if it was days ago then it may also be a clue that cron has been running, but has since stopped. If anyone has an alternative approach, please let me know I’d like to hear it.

Posted on September 6, 2010 September 6, 2010

New in Cloud Backup: Platform Independent Magento Backups

This is just a quick note to say there is a new version of the Cloud Backup extension now available on Magento Connect and also for direct download and manual installation – it is version 0.2.2. Our Magento Offsite backup beta progam has been great so far with many participants offering useful feedback and being very willing to help us debug issues.

In the first week we have been able to determine that:

Different hosts have very different security settings that make taking full tar archive backups quite tricky.
Windows hosting users cannot rely on a command-line tar application. If you are a WAMP or WIMP user please try it out and let me know if it works ok on your host.

As a result I have changed from a system call to tar to using the PHP class Archive_Tar to tar the files in a what will hopefully prove to be a platform independent manner.

I have also added the ability to exclude certain root directories from the backup. This is mainly as a way for users where web server permissions prevent access to certain Apache owned files by the user. For example you could exclude var if all of the session and cache files are Apache owned and unreadable.

The way to exclude files is to list them comma-separated in the exclude text field. Leave it blank to exclude nothing. If you notice backups failing due to files in a certain directory, try excluding it. You could always petition your web host for relaxed permissions in order to include the directory again. (Or move to a better Magento host!).

As always please feel free to let me know if you have any ideas for the development and report any issues – we’re happy to look into them. If you’re not already backing up your Magento store, why not try our cloud backup?