RSync Files to a Unix/BSD Backup Device from your Mac Laptop

My photo-taking workflow while on vacation usually involves taking a lot of photos daily, dumping them to a laptop, processing, then backing them up once I have returned home.
Previously, I accomplished this manually using BeyondCompare for Windows, as that would run on Windows Home Server.
Since moving to ZFS-based storage, however, this is no longer an option as BeyondCompare only has a Linux client (nothing for Unix/BSD).
There are other ways to get around this:

  • SSHFS and Meld – Complicated, somewhat bloated, but great BeyondCompare alternative
  • *Commander Utilities – Midnight Commander derivatives can accomplish similar tasks using the ctrl+x,d shortcut
  • Rsync – typically installed by default, easy to script

I chose Rsync as I wanted something more automated, but I do find myself using Midnight Commander from time-to-time to simply “get things done” when syncing files other than my images.

Here’s how I did it:

rsync -a -e ssh /volumes/PICTURES/ 'username@mymac:/Volumes/BIGRAID/'

Let’s break this down into smaller pieces:

rsync – this is the command that will do our heavy lifting and file comparison

-a – archive mode

-e – specify an RSH replacement

ssh  – use SSH

/volumes/PICTURES/ – this specifies the “Volumes” folder on my Mac, and the “PICTURES” drive within it. Replace this with the location of your items to backup

 – note the use of single quotes here. We’re using these in case there are spaces in the folder names, and we could have done the same above.

username@mymac – We’re logging on to the host “mymac” with the username “username”. You’ll probably want to change these. I use a hostname here, but you could just as easily use an IP address if you use static IP addresses.

:/volumes/BIGRAID/ – the colon denotes a subfolder on the server we are backing up to, and /volumes/BIGRAID in this case refers to a ZFS pool called “BIGRAID”.

Do you have a similar backup strategy for BSD/Unix targets that you would like to share?

ZFS Build Checklist

I’ve decided to replace the Windows Home Server Vail server with something capable of handling newer builds of ZFS and the inherent deduploication.

Here’s a quick kit list and build diary I’ll try to keep up-to-date as I go along.

Kit:

  • Dell Perc6i – this is essentially a port multiplier. I scored it from eBay on the cheap, though it was delivered from Israel, took awhile, and had neither cables nor mounting bracket.
  • OCZ RevoDrive 120GB – Though the RAID controller on this card is not supported in Linux/Solaris, the drives show up as two separate devices as long as you make sure to put it in the right PCIe slot. That means it’s perfect for both ZIL (log) and L2ARC (cache).
  • 2x Intel 80GB X25-M SSDs – these will house the virtual machine files to be deduped. Very reliable drives, and though they might not be the fastest in terms of writes, the speeds are relatively constant which is quite handy compared to solutions that attempt compression like SandForce controllers. ZFS will take care of that, thanks.
  • (IN TRANSIT) 2x Dual Port 1gbit Intel PCIe NICs – I’ll use these for the direct connection to the virtual machine host. Currently one link is used, but when reading from the SSD drives the line is saturated.
  • (IN TRANSIT) 32 Pin SAS Controller To 4x SATA HDD Serial Cable Cord – This is needed to plug in 8 drives to the LSI controller.
  • 5x 1.5TB Seagate hard drives – These will be the bread-and-butter storage running in RAID-Z2 (similar to RAID 6).
  • 3x 3TB Seagate hard drives – These might simply be a large headache, but the plan was to have an extra 3TB RAID-Z2 for backups in another machine. Unfortunately there seem to be issues with drives that are 4k presenting themselves as 512b. I may be able to get around this by hacking or waiting as they become more popular. For now 2 of them are in software RAID1 on a Windows 7 host, and the other remains in the external USB 3 case and is used as a backup drive.
  • NetGear GS108T Switch – A cheap VLAN-capable switch should I decide to use more than 2 bonded ports (I doubt it), currently running the lab.

blandname forum open!

I haven’t been getting much feedback on the latest poll, and it’s probably my fault because it’s positioned under the adverts… oh well.

As a lazy developer and software-lover, my only choice is to implement ALL of the SEO Services Sydney solutions, one at a time.

So that’s how we got here with our second real announcement, after the Digg plugin for Pandion (thanks to 43things for reminding me once weekly that I needed to start an open-source project).

I’ve opened a forum for all to use – you can feel free to post tech support questions, links you have found, or just general chit-chat if need be. I’ve set up a pretty good spam filtering system, but we’ll see how it works. I honestly have a hard time believing anything could work as well as Dr. Dave’s Spam Karma 2 plugin for WordPress.

So head on over to the blandname forum and check it out!

I’ve set up BBPress there, and to be honest it was a cinch. I’ll post a how to do seo in montreal later, but it’s not so involved. I was able to get modrewrite going right away – which in my opinion is an absolute necessity for proper SEO. I will be attempting to integrate the themes and users via MySQL shortly, so if you’ve registered with blandname, you’ll have access to the forums as well.

Distro of the week: Berry Linux

Up until a few years ago I was getting fed up trying to show Japanese friends that linux was a viable option to Windows XP SP2, because the input method that Linux used at the time quite truthfully stank. The conclusion was that with an English keyboard, Windows was the way to go because of the (then) fancy Japanese input method applet.

Later on, I found out about Berry Linux, a Japanese LiveCD distro based on RedHat/Fedora that can easily show off Linux and allow users to do things like compose email that Japanese people can actually read (common problem on Windows), and surf the web in Japanese by default, instead of having to manually choose codepages and the like, which frustrates must computer users (if they haven’t already given up).

Berry Linux got an update today (to version 0.75), and now supports fancy-pants XGL – which I’m sure you’ll agree kicks Aero’s behind quite handily (footily?). If you’re trying to run Berry Linux Mini as a virtual machine you’ll get a console instead of X and you’ll lose the XGL capabilities, so I’d recommend testing it on a physical machine instead of a virtual machine – it’s a LiveCD so it’s quite safe to do so.

Berry LinuxBerry LinuxBerry Linux

Here’s a feature rundown taken from the Berry Linux site’s English page (corrected a few typos):

    Common Features:

  • Support for Kernel-2.6. ALSA, ACPI, selinux.
  • Overlay Filesystem Support.
  • XGL, 3D Desktop, support.
  • Berry Linux is not necessary to install. (Root partition is in the RAMdisk using initrd, all commands are operated by being transfered from CD-ROM)
  • It can install on Windows without parting partitions to use Setup.exe or install.bat. (Using squashfs/cloop/loopback device)
  • It can install to your hard disk on Linux. (Use Berry Linux Installer or Copy under /BERRY/, and set up LILO or grub)
  • Berry Linux can boot from USB-HDD/memory.
  • Berry Linux uses WHIZ, a very sharp Kana-Kanji conversion system. (WHIZ Project)
  • If you push the windows key, and show the K-Menu.
  • Automatically recognizes USB storage, and show icons.
  • Berry Linux uses free Japanese True Type Fonts.
  • Berry Linux uses bootsplash when booting.
  • Berry Linux uses DHCP to connect the Network. (If you’d like to use PPPoE, you should setting up it on the Terminal)
  • It’s possible to save personal setting.
  • Red Hat Fedora compatible.
  • Using new technologies.
    Berry Linux’s Features:

  • You can listen to mp3 using BMP/XMMS, and play DVD/DivX using MPlayer, XINE and Kaffeine.
  • You can edit files of Microsoft Word, Excel by OpenOffice(TM).
    Berry Linux Mini’s Features:

  • Minimum Linux environment is available by using Berry Linux Mini.
  • Its file size is very small. (148.0MB)
  • It’s light. Because of the Window Manager is Fluxbox.
  • You can enjoy comfortable Linux environment.
    Berry Linux Server’s Features: (Published Berry Linux Mini as alpha version)

  • Build the Linux Server easily.
  • Minimum Server Linux enviroment is available by using Berry Linux Server.
  • Its file size is very small. (161.3MB)
  • It’s light. Because of the Window Manager is WindowMaker.
  • You can enjoy comfortable Linux enviroment.

    To do:

  • Support Software Suspend.
  • Support Captive NTFS.

You can grab Berry Linux from the SourceForge page or the Berry Linux main page – but I’d recomend using SourceForge if you are in North America as it is much faster.

Swik stole my content!

Update 2: After contacting one of the Swik developpers, I was able to have them clear the site of my content. Though they still appear to be syndicating my content, at least the older entries are gone, and I have since edited the RSS feed properties so that only 100 characters are fed. Lesson learned, the hard way.
Update:This article has now been republished on the Swik website. I’m speechless.

Swik Update

It’s said that imitation is the sincerest form of flattery. One could suppose that blatant content stealing would make you quite flattered indeed. Once would be OK, but having half of your websites content mirrored elsewhere is something else altogether.

It seems there’s a new splog (spam blog) on the block, and it’s being funded by none other than SourceLabs.

I found out about Swik while doing a vanity search for blandname.com, and was flabbergasted.

It seems the kind folks at SourceLabs now think it’s OK to republish full websites on their Swik websites, without so much as asking. Have a look at the evidence – blandname has half of it’s content republished at Swik, sometimes 3 times over.

I’m assuming that they simply are grabbing del.icio.us links and RSS feeds that contain the keywords “open source” – but they could at least limit what they republish to 50 words or so. But no, they take it one step further and hotlink to your images, videos, and audio files as well. Way to go you bumbling fools.
I’ve sent them a kind but stern email letting them know this isn’t so cool, and I’d recommend that anyone else affected look into this as well.

Here’s a recent example that was republished at Swik 3 whole times:

blandname swik

I truly appreciate what they are trying to do, but lack of testing, and not asking people if it was OK to republish their content is not very nice. But I digress.

The real issue at hand is that Swik claim that all content on their website is Copyleft, which this site clearly is not. In case they missed it, I have modified my RSS feed leaving them a note expressly stating the fact…

Installing WordPress MU on a Dreamhost Server

Blandname is currently hosted with DreamHost, and we’ve been here for years. It’s cheap, offers lots of goodies, and one-click installs allow us to easily install and test web-based software. Not to mention that they also support Ruby on Rails, and give you SSH access and the ability to run a Jabber server as well as unlimited MySQL databases. If you are wanting to host your own website then choose a UKservers.

You’ve also probably gathered that blandname is currently running WordPress. Dreamhost Circulo Marketing has had a one-click install for WordPress just like ipage has for a while now, and since it was handy at the time, we went for it.

But things change, and one-click installs often are not enough to satisfy most webmasters, which is how we got where we are today. Since my goal with blandname is to create another multiuser blog similar to what has already been running for years at yottabite, but instead of having one big weblog, we’d like to have multiple subdomains like string.blandname.com, which WordPress MU allows you to accomplish, automatically.

Unfortunately DreamHost doesn’t support WordPress MU‘s subdomains by default yet (you can always send them an email), but we can still get away with subfolders, which is more than good enough for a test.

This guide will require familiarity with DreamHost’s control panel, as well as common Bash shell commands as we will be using SSH.

The first step is to make a test domain for you WordPress MU install. In my case, I navigated to the “Domains” section of the left-hand menu, then to the “Manage Domains” section of the DreamHost panel, and created the new subdomain test.blandname.com. You’ll want to make sure to select PHP5, and enable extra security. This typically takes about 10 minutes to complete, but we still have the database to add, so let’s get to that at the same time.

In the “goodies” section of the DreamHost control panel, select “Manage MySQL”. The default view is to set up a new MySQL database, which is what we’re going to do. Create a unique database name, the subdomain you would like it to use, as well as the data base username and password. Make sure to keep note of all of these settings as we will need them when installing WPMU.

DreamHost will have by now created a folder in your SSH root that will allow you to place files there and start some of the work while we wait for the subdomain to be created and propagate. Login to your server using SSH (you’ll need to use either your DreamHost hostname here or another web address for now – you can use the WordPress Mu domain later). Now we’ll navigate to the new subdirectory that was created when we setup the new subdomain by typing: cd test.blandname.com Change the folder name to whatever is pertinent in this case.

Now that we’re in the correct folder, we’ll grab the latest using the always-handy WGET. Here’s the code:

wget http://mu.wordpress.org/latest.tar.gz

gunzip latest.tar.gz

tar -xvf latest.tar

cd wordpressmu-1.0 (this will probably change, ls -al will tell you the dirname)

cp -rf * /home/YOURUSERNAME/test.yourdomain.com/

cd ..

rm -rf wordpressmu-1.0/

Now we’ve got a clean directory structure in the root of our test domain, and we’re set to go ahead with the WordPress MU installation.

By now the subdomain has probably propagated because DreamHost is getting faster and faster, so using your web browser, navigate to test.yourdomain.com

Next you’ll want to retrieve the soiled napkin, SubEthaEdit file or whatever else it was that you used to jot down the database settings, and plop them in here. They are very straightforward, and this is typically the most problematic so check them twice but have no fear: if you mess up WP MU will tell you, and you can retrieve the settings from the “Manage MySQL” section in the DreamHost web control panel.

The rest is quite simple: you’ll be met with a typical WordPress installation page, but instead it’s for WorPress MU. The first question that needs to be asked is whether or not WordPress MU users will be using subdomains or subfolders of the root WPMU installation. As previously stated, DreamHost currently does not support subdomains by default (I’ve put in a request, here’s hoping), so we’ll select subfolders here. WP MU will have already placed the domain name you will be using in the yellow textfield, but if you had decided to use subfolders instead of the webroot, you’ll want to specify that here as this will affect all links as well as your RSS feeds.

Lastly, we’ll want to name our multi-user WordPress MU blog, and specify the email address that you will use for things like spam reports, and replies to your comments on the parent blog.

Click on that small “submit” button, and let’s see what happens!

Hopefully on the next screen you’ll see this message:

Creating Database Config File: DONE
Congrats! Your WPMU site has been set up and you have been sent details of your login and password in an email.

Click on the link provided, and get with customization, as we’re all done.

Firefox 2 – Display Single Close Tab Button

Close Buttons on Tabs in Firefox 2.0

If you’re now using Firefox 2.0, you’ve probably already noticed that tabs now have close buttons by default, instead of having a single tab close button on the far right-hand side of the toolbar. If you preferred the older behaviour, simply navigate to the “about:config” page (type it in like a URL or webpage address), and find the section marked “browser.tabs.closebuttons”. Double-click on this entry, and set the value to 3 to display a single close button at the end of the tab strip (Firefox 1.x behavior).

Alternatively, you can set the value to 0 to display a close button on the active tab only, 1 to display close buttons on all tabs (Default), and set it to “2 to not display any close buttons at all.

taken from the about:config page at MozillaZine

A parting tip: To close the current tab using your keyboard, press CTRL+W. If you close a tab by mistake, just head to History > Recently Closed Tabs, and you can resume work from there (alternatively, you can use the arcane keyboard combo CTRL+SHIFT+T).

If you prefer using your mouse and want an even faster way about it – try the middle-mouse click (that’s typically a scroll button). When you click the scroll on any tab it instantly closes the tab for you. I have been using this method quite a bit lately. Though it seemed quite alien at first, I quickly got used to it and it’s now a major part of my Firefox 2 workflow.

Beyond Firefox 2 – Testing Minefield (Firefox 3)

OK, so everyone is very excited about Firefox 2. I’m excited, your excited, all the social networks are ablaze.

But development does not stop at whole numbers, and the testing must go on – which is how we got here.

So I’m testing Firefox 3 (Minefield). That’s right, I won’t be outdone. I must run the most unstable software. Well at least on the test boxes. So currently the Vista, Leopard, and Edgy boxes are running “firefox-3.0a1”.

The first things you will notice are the graphics – everything has a nice sheen to it (in fact, looks just like Firefox 2!), while remaining true to the normal Firefox UI. Also of note is the fact that favorites are now run using a SQLite3 database. For the full list of features, head on over to the Burning Edge page.

Firefox 3 Minefield Toolbar
Firefox 3.0a1 Toolbar
Firefox 3 Minefield Tabs
New Tabs in Firefox 3.0a1 Minefield

As far as I can tell this is just as stable as Bon Echo was (that’s right – was stable for me), with inline spell-checking and some other fancy goodies.That’s all for now, some more screenshots from other platforms (other than Windows XP of course) and crash reports as they happen

Note: of course this is somewhat tongue-in-cheek. I’m sorry for the Mozilla devs. We all need to stop worrying about the cutting edge and appreciate what we have – fantastic browsers! Opera, Safari, IE7 and Firefox are all great, really.

Digg Plugin for Pandion

What it is:

How it was written:

  • Digg for Pandion was based on the Slashdot plugin for Pandion, with added Base64 Digg icons, and a bit of CSS.

Todo:

  • Add Digg buttons so you can Digg up articles of interest, or comment on them.
  • Allow users to select different Digg feeds that the plugin could display.

Click here to download the blandname Digg plugin for Pandion (0.1-alpha)

How to install the plugin:

  • Unzip the file and simply copy the plugin to the Pandion plugins folder. The plugin will create all the necessary files needed at runtime automatically.