Friday 25 March 2011

A Backup Scheme Described

I wrote a bit about my backup routine in my rant about TrueImage. I wanted to set out in more detail the scheme I have set up because it also incorporates some mklinking referred to in my previous post.

My previous example of mklinking dealt directly with steam applications. There is another type of data that you do not want to have clogging up and SSD. That is any large file that you only require to access sequentially. Or in non-computer science lingo, music, videos and photos. These can be large files (a modern SLR camera will happily spit out 10Mb image files), but you do not need the super fast access time of an SSD to use them properly.

Think about it, you could happily watch a movie from a DVD player. That has a PATHETIC data rate and seek time compared to an SSD, but it works fine for movie file. Movie files are also LARGE. My Sanyo camcorder mentioned elsewhere spits out files up to 20mins/Gb. They take up a lot of space. If a 64Gb SSD (with usable space of say 60Gb) was used exclusively for these video files you would fit no more than 20 hours of video on it. For £60. That's £3/hour for storage of video. That makes no sense. Not when at the time of writing you can pick up a 2Tb classic drive for the same £60. OK, that 2Tb drive is only actually about 1.8 real Terabytes. Still, that's 620 hours of video for £60. That's £0.09/hour.

So for large data files we want the 9p slow and steady storage, not the £3 super fast storage. What I put on the classic disk is basically the contents of My Videos, My Music, My Pictures, and My Documents. To do this I create a directory on the classic disk with appropriately named folders in it. If your OS is on the SSD, then your classic disk is probably mounted at d:. You can do this with the mouse and gui, but the dos prompt commands (windows key + r, type cmd, hit enter) look like this:

cd d:\
mkdir Data
cd Data
mkdir Videos
mkdir Music
mkdir Pictures
mkdir Documents

For the benefit of the hard of thinking, those commands [c]hange to the [\] root [d]irectory of the [:] disk assigned the letter [d]. They next [m]a[k]e a new [dir]ectory, [c]hange into that new [d]irectory, and [m]ake some other [dir]ectories.

You then fill up those folders with the data of choice. Copy this from your non-SSD old system drive, or from your up to date back up. Then, if using windows 7, you need to delete the Videos, Music, Pictures, Documents from your user folder. There should be nothing in those folders on a fresh install - but check anyway, and if there is then move it to the folders on the classic disk we just made.

cd C:\Users\[your username here]
xcopy Videos D:\Data\Videos /E /H /K
rmdir Videos /S
xcopy Music D:\Data\Music /E /H /K
rmdir Music /S
xcopy Pictures D:\Data\Pictures /E /H /K
rmdir Pictures /S
xcopy Documents D:\Data\Documents /E /H /K
rmdir Documents /S

[xcopy] is a special dos command that is used to copy large collections of files at once. It is basically more flexible and powerful than the standard dos [copy] command. For our purposes it is exactly the same as Control+c'ing a folder and Control+v'ing that folder somewhere else. The [/E] flag tells it to copy all subfolders even if they are empty, the [/H] flag makes sure it copies hidden or system files, and the [/K] flag copies the attributes of the files being copied (read only etc) as well. The other command [r]e[m]oves the specified [dir]ectory and all its [/S]ubdirectories.

Finally you want to set up the mklinks:

mklink /J "C:\Users\[your username here]\Documents" "D:\Data\Documents"
mklink /J "C:\Users\[your username here]\Videos" "D:\Data\Video"
mklink /J "C:\Users\[your username here]\Pictures" "D:\Data\Pictures"
mklink /J "C:\Users\[your username here]\Music" "D:\Data\Music"

You should now be up and running. If you go to the start button and click on Music, the folder that should open is the one on the classic disk. The benefit to using mklink is that it appears to the OS and all software running on it (such as iTunes) that all this data is in your user folder on the SSD, so you do not need to specially configure any other software. It should just work.

Now you have set that up, backups are a doddle. Buy a large external 3.5" disk that comes with it's own power supply. Yes, it is bulky. Yes it is a pain to have to carry around a data and power cable. BUT it is much cheaper per Mb than a 2.5" USB powered external disk. And all it is going to do is sit in the vicinity of your main machine ready to be used as a backup device at a moments notice. You then just need Allway Sync, or a similar folder syncing application, to sync the d:\data folder onto your external drive. Make sure you sync that ONE folder every so often (i.e. when you have bought new iTunes music, written a new document, or uploaded a camera card to your desktop) and you have a backup copy of ALL of your data.

I find the advantages of this system are:

Simplicity. The backup is just a copy of all your data files. You can plug the external disk into another machine running virtually any desktop OS and your have instant access to your files.
Cost. Folder Synchronisation software is inexpensive (Allway Sync is nagware).
Speed. All I do is open Allway Sync, choose the "Data" profile I have set up, run an analysis just to check, and then run a sync. It does not take a long time unless you have added a crap load of stuff.
Flexibility. Running dual OS's? Simply link Ubuntu's similarly named data folders to the ones on the windows disk. Hey presto, same data, two different OS's.

There are disadvantages to this system:
Versioning. The backup is a simple mirror of your files. It does not keep multiple copies of your files, so you cannot undo a change, or deletion, once you have synced. You could implement this, but you would need more expensive dedicated backup software. As far as I am concerned, I do not want to lose family media. I am not overly exercised about restoring last Tuesday's copy of a 1Gb video file rather than last Wednesdays. Generally the data in these folders is NOT going to change, it is going to be added to or deleted, but not altered.
Onsite. The backup disk is in the same room as the desktop with the original data, which is not going to help if the equipment is stolen, or the building burns down. For that protection, you need to pay for a online backup solution - or get an ISP who provides one.

I prefer the advantages to the disadvantages. I do see a need for a versioning backup system, which is why I bought TrueImage Home. But I use that pretty exclusively for the OS disk. With that you absolutely want to be able to go back to last Tuesday, or "whenever I didn't have this bastard virus".

You may also want to move some other folders from your user folder on the SSD to the classic disk. For me a priority was the Downloads folder. You delete the original folder as normal and then:

mkdir d:\Data\Downloads
mklink /J "C:\Users\[your username here]\Downloads" "D:\Data\Downloads"

You probably do not want to backup the downloads folder - I use mine as a scratch area for downloading loads of rubbish. If I want to keep stuff I download, I shift it to an appropriate data folder. So you will now want to edit the profile you have set up in Allway Sync to EXCLUDE the Downloads folder from the backup. This saves space on your external disk.

You can also get cute with the exclusion filters INSIDE the folders you are syncing. So if you have a Temp scratch area in Music or Videos, you can exclude that from the backup as well.

No comments:

Post a Comment