RSync Files to a Unix/BSD Backup Device from your Mac Laptop

My photo-taking workflow while on vacation usually involves taking a lot of photos daily, dumping them to a laptop, processing, then backing them up once I have returned home.
Previously, I accomplished this manually using BeyondCompare for Windows, as that would run on Windows Home Server.
Since moving to ZFS-based storage, however, this is no longer an option as BeyondCompare only has a Linux client (nothing for Unix/BSD).
There are other ways to get around this:

  • SSHFS and Meld – Complicated, somewhat bloated, but great BeyondCompare alternative
  • *Commander Utilities – Midnight Commander derivatives can accomplish similar tasks using the ctrl+x,d shortcut
  • Rsync – typically installed by default, easy to script

I chose Rsync as I wanted something more automated, but I do find myself using Midnight Commander from time-to-time to simply “get things done” when syncing files other than my images.

Here’s how I did it:

rsync -a -e ssh /volumes/PICTURES/ 'username@mymac:/Volumes/BIGRAID/'

Let’s break this down into smaller pieces:

rsync – this is the command that will do our heavy lifting and file comparison

-a – archive mode

-e – specify an RSH replacement

ssh  – use SSH

/volumes/PICTURES/ – this specifies the “Volumes” folder on my Mac, and the “PICTURES” drive within it. Replace this with the location of your items to backup

 – note the use of single quotes here. We’re using these in case there are spaces in the folder names, and we could have done the same above.

username@mymac – We’re logging on to the host “mymac” with the username “username”. You’ll probably want to change these. I use a hostname here, but you could just as easily use an IP address if you use static IP addresses.

:/volumes/BIGRAID/ – the colon denotes a subfolder on the server we are backing up to, and /volumes/BIGRAID in this case refers to a ZFS pool called “BIGRAID”.

Do you have a similar backup strategy for BSD/Unix targets that you would like to share?

Clear All ZFS Snapshots

If you’ve been running snapshots for a while and have already backed them up, you might occasional need to delete all zfs snapshots for your pool.
Typically, you’d do this as part of your backup script, assuming that they have been written correctly.

First, to find the used snapshot space, run this command:
zfs list -o space
This will give you a detailed readout of your pools and snapshot space used.

Here’s my script to wipe ZFS snap shots, but I am certainly open to suggestion:
zfs list -H -o name -t snapshot | xargs -n1 zfs destroy
Again, caution is needed as this will remove ALL SNAPS from your pools.

The 10 Best VMware Virtual Appliances

As suggested in the comments, I’ve updated this post here: http://blandname.com/2012/04/09/top-10-virtual-appliances-revisited/

Daniel and Bitnami have quite a few of these already published, which is pretty cool! 

This list is subjective, and you’ve been warned!

All of these virtual appliances have been tested with ESX server, and may have issues elsewhere.

For appliances that needed it, I used R3 Data Recovery VMware Converter, the version that ships with Virtual Infrastructure 3.5 (VI3.5).

Please note that both ESX 3.5 and Virtual Center 2.5 are available as trials from VMware currently, and I would highly recommend trying them out as it really is night and day compared to VMware Workstation, Server and Player.

That said, for the most part you’ll be fine working with VMware Server 2.0 – it’s free and has a special version of VMware Infrastructure Client to boot.

The list:

  1. Astaro Security Gateway – This is a must in any build for me. I use this to bridge between my LAN/WAN and the virtual networks that I create. There is a 10-device, 1000 connection “home user” license available from My Astaro that should be more than sufficient to get you up and running with a clean, secure virtual network.
  2. Ubuntu 7.10 JeOS Mini-image – this image weighs in at only 70MB or so, expands to roughly 200MB, has apt-get installed, and is a perfect candidate for building virtual appliances with. VMware tools is installed, so you don’t need to worry about things like date and time sync.
  3. OpenBSD 4.2 – The OpenBSD image is great for getting started in the OpenBSD world: learning the shell, commands, networking, and in my case, firewalling. The verison I use comes from Chrysaor.info, but feel free to use your own.
  4. OpenSuSE 10.3 – I can’t live without this virtual appliance – I use it for just about everything, and is the first appliance installed in any environment. Note that it is a bit bloated, containing USB, sound and other components typically not needed in a virtual environment. On the other hand, since it’s tried and tested on my end, it’s a lifer.
  5. Trac – I use Trac as a wiki and VM staging log. I consider all VMs, hosts and Virtual Center as software projects, and monitor changes closely. If ever I need to pull up quick info about a virtual machine, host, network, router or firewall, it’s all in Trac.
  6. WordPress – I use my WordPress virtual machine to stage different versions of blandname, to test updates, upgrades, and plugins. This also allows me to change themes, move Adsense blocks around, and generally to play without fear of losing revenue or breaking something.

Continue reading