ZFS Build Checklist

I’ve decided to replace the Windows Home Server Vail server with something capable of handling newer builds of ZFS and the inherent deduploication.

Here’s a quick kit list and build diary I’ll try to keep up-to-date as I go along.

Kit:

  • Dell Perc6i – this is essentially a port multiplier. I scored it from eBay on the cheap, though it was delivered from Israel, took awhile, and had neither cables nor mounting bracket.
  • OCZ RevoDrive 120GB – Though the RAID controller on this card is not supported in Linux/Solaris, the drives show up as two separate devices as long as you make sure to put it in the right PCIe slot. That means it’s perfect for both ZIL (log) and L2ARC (cache).
  • 2x Intel 80GB X25-M SSDs – these will house the virtual machine files to be deduped. Very reliable drives, and though they might not be the fastest in terms of writes, the speeds are relatively constant which is quite handy compared to solutions that attempt compression like SandForce controllers. ZFS will take care of that, thanks.
  • (IN TRANSIT) 2x Dual Port 1gbit Intel PCIe NICs – I’ll use these for the direct connection to the virtual machine host. Currently one link is used, but when reading from the SSD drives the line is saturated.
  • (IN TRANSIT) 32 Pin SAS Controller To 4x SATA HDD Serial Cable Cord – This is needed to plug in 8 drives to the LSI controller.
  • 5x 1.5TB Seagate hard drives – These will be the bread-and-butter storage running in RAID-Z2 (similar to RAID 6).
  • 3x 3TB Seagate hard drives – These might simply be a large headache, but the plan was to have an extra 3TB RAID-Z2 for backups in another machine. Unfortunately there seem to be issues with drives that are 4k presenting themselves as 512b. I may be able to get around this by hacking or waiting as they become more popular. For now 2 of them are in software RAID1 on a Windows 7 host, and the other remains in the external USB 3 case and is used as a backup drive.
  • NetGear GS108T Switch – A cheap VLAN-capable switch should I decide to use more than 2 bonded ports (I doubt it), currently running the lab.

The 10 Best VMware Virtual Appliances

As suggested in the comments, I’ve updated this post here: http://blandname.com/2012/04/09/top-10-virtual-appliances-revisited/

Daniel and Bitnami have quite a few of these already published, which is pretty cool! 

This list is subjective, and you’ve been warned!

All of these virtual appliances have been tested with ESX server, and may have issues elsewhere.

For appliances that needed it, I used R3 Data Recovery VMware Converter, the version that ships with Virtual Infrastructure 3.5 (VI3.5).

Please note that both ESX 3.5 and Virtual Center 2.5 are available as trials from VMware currently, and I would highly recommend trying them out as it really is night and day compared to VMware Workstation, Server and Player.

That said, for the most part you’ll be fine working with VMware Server 2.0 – it’s free and has a special version of VMware Infrastructure Client to boot.

The list:

  1. Astaro Security Gateway – This is a must in any build for me. I use this to bridge between my LAN/WAN and the virtual networks that I create. There is a 10-device, 1000 connection “home user” license available from My Astaro that should be more than sufficient to get you up and running with a clean, secure virtual network.
  2. Ubuntu 7.10 JeOS Mini-image – this image weighs in at only 70MB or so, expands to roughly 200MB, has apt-get installed, and is a perfect candidate for building virtual appliances with. VMware tools is installed, so you don’t need to worry about things like date and time sync.
  3. OpenBSD 4.2 – The OpenBSD image is great for getting started in the OpenBSD world: learning the shell, commands, networking, and in my case, firewalling. The verison I use comes from Chrysaor.info, but feel free to use your own.
  4. OpenSuSE 10.3 – I can’t live without this virtual appliance – I use it for just about everything, and is the first appliance installed in any environment. Note that it is a bit bloated, containing USB, sound and other components typically not needed in a virtual environment. On the other hand, since it’s tried and tested on my end, it’s a lifer.
  5. Trac – I use Trac as a wiki and VM staging log. I consider all VMs, hosts and Virtual Center as software projects, and monitor changes closely. If ever I need to pull up quick info about a virtual machine, host, network, router or firewall, it’s all in Trac.
  6. WordPress – I use my WordPress virtual machine to stage different versions of blandname, to test updates, upgrades, and plugins. This also allows me to change themes, move Adsense blocks around, and generally to play without fear of losing revenue or breaking something.

Continue reading

Learning VMware ESX 3.5 on the Cheap

Lately I’ve been scouring the web for used gear because there seems to be an influx of incredibly powerful stuff at amazing prices.

This all came about with me wanting to learn ESX 3.5 and how to build profitable online business, and needing the hardware as well as the network to run a feasible set up, with DRS, HA and Storage Vmotion. And I did it – on the cheap.

The first thing that you should know about ESX 3.5 is that it now works with many non/budget-RAID SATA chipsets, though not supported. Two that are readily available are Intel’s ICH5 and Silicone Image’s Sil SATA line. This typically depends on the BIOS you are using, but in regards to the ICH5, you’ll want to disable IDE compatibility mode, and as for Sil – you’ll sometimes want to turn on the RAID (though some Sil single channel cards also work, ie the Vantec SATA 1-port).

The second is that drive space is inexpensive. A 500GB SATAII Western Digital drive will only set you back about $100 CAD/USD. Two of these gives you redundancy! Combine the cheap drives with software like FreeNAS or OpenFiler, and you have yourself a 1TB iSCSI NAS for a fraction of the cost/MB of larger solutions. Just don’t pretend it’s foolproof… With any proper iSCSI, you’ll want some nice and spiffy ethernet cards, and in my case I used the tried-and-true Intel Pro 100 successor, the Pro 1000. You can find the Pro1000 GT for roughly $40, and a PCIe version of similar capabilities for about the same amount. The PCI version is compatible with VMware ESX 3.5, OpenFiler, FreeNAS, and Windows Server 2008. I hear the same goes for the PCIe version, and I will be able to let you know shortly.

Since we’re on the topic of networking, you’re going to either want two gigabit (gbit) switches, or a nice gbit switch with more ports and VLAN ability. I lucked out, and got a used 3Com SuperStack III (3C17706) for next to nothing. I’ve seen plenty of somewhat lesser-known (but just as nice) gigabit HP Procurve, Extreme Networks, and even Dell gear at plain stupid prices on ebay and Craigslist. Seriously. I’m talking 50$ for a 24 port 10/100/1000 switch! The trick on ebay is to not bid at all on stuff until it is about to end… then just pick it up. Well I guess everyone has figured that out by now, but it still works. Don’t draw attention to it by watching it like a hawk – just set up instant messaging reminders, and swoop in. As for Craigslist, I have RSS feeds for things I am interested in: 1U, 2U, 3U, 4U, 6U, rack, rackmount, etc. I check these on a regular basis, and make sure to email right away. Craigslist people are friendly, but will typically sell to the person who a) emails first, b) offers to pick it up the soonest, and c) doesn’t give them a hard time.

Now we’re into routing, mostly because I want to talk about it. This setup does not require any routing at all, but it’s a bit better to have a protected connection to the internet. My personal opinion here is to avoid Cisco at all costs, as recently the re-licensing has made buying one used a lot more expensive than in the past. That said, I do, in fact, own a Cisco router – what can I say! It’s like the gold standard. Of course my opinions are my own, and not that of my employer. For a cheaper routing solution, look to used Juniper, SonicWALL, and even open source stuff like Astaro (which also happens to run in VM…) Peronsally, I run a home licensed Astaro ten user virtual machine, a Juniper 5GT wireless, and barely use the Cisco 2611. That’s just me. If you’re having a hard time finding the Astaro licensing, just let me know, but rest assured it does exists, and is perhaps the EASIEST way to turn on VPN so you can have access to your virtual lab anywhere.

Alright, now we’re at the meat and potatoes – CPU and memory, the power behind all of this. Now, if power isn’t really a big deal, but you want to learn the cool features like the afore-mentioned DRS, HA and Storage Vmotion, the main thing you will want is memory. I’ve found that buying enthusiast RAM on Craigslist is VERY easy. Pick a brand like Crucial, OCZ or Kingston HyperX, and you’re bound to have masses of kids who all read the same articles, and are all selling the same RAM used, pretty much at the same time. Watch the trends, and you can easily build 4 boxes running over 3GB or RAM each, for cheap. Dirt cheap.

If you’re going the consolidation route, your best bet (used) is an Opteron solution. While you can find Opteron 185 and 175 chips on ebay, I find that they are asking a fortune for them because they are socket 939, and are considered top of the line for the specific platform. If you opt for the 165 dual core version, you can use cheap enthusiast RAM with great timings, and get a pretty good processor at the same time. It also means that you can get a motherboard to support both pretty much anywhere at bargain basement cost, and one that will typically have a Sil SATA chipset, or you can add one later.

In the case of the multi-box scenario – I’m using 4 Intel P4 3.0GHz HyperThreading processors. Not powerhouses by any means, but when it comes to storage, you can get Intel P4 motherboards that have ICH5 chipsets very easily because they well so prolific at the time – just make sure to watch those temps.

I think that kinda sums it up, as far as a used, cheap, VMware ESX 3.5 lab goes.

If you have any questions, feel free to shout them out.

VMware Releases VI3 Perl Toolkit Virtual Machine

Recent stock market darling VMware has just released a Perl toolkit for it’s Virtual Infrastructure virtualisation product, as well as a Perl toolkit virtual machine that you can download for free to play around with.

VMware describes the toolkit as “an easy-to-use Perl scripting interface to the VMware Infrastructure API (VI API). Administrators and developers who may be more familiar with Perl (rather than with Java, C#, or other programming languages) can readily leverage the VI API. For developers who have previously worked with the Scripting API (VmPerl API), the VI Perl Toolkit is the tool of choice.”

An example VI3 Perl script, perf.pl, can be downloaded at the VMware forums site. Perf allows you to measure the performance of your virtual machines running on ESX 2.x or 3.x servers during a specified period of time.

Free Download – VMWare Introduces Virtual Lab Manager Beta

VMWare announced that they will be opening the Virtual Lab Manager to the public today, and the software is now available for download (ESX 3.0.1 too!)

Virtual Lab Manager

Virtual Lab Manager is a product for managing virtual machines running on ESX 3 or VMWare Infrastructure servers, and allows you to do nifty things like save SIDs, MAC and IP addresses, and deploy groups of configurations to multiple machines.

Here’s a full rundown of the features from the datasheet:

Multi-Machine Configurations
• Create multi-machine configurations in seconds using machine templates – no limit on machine count, no manual adjustments
• View configurations in use with thumbnail console views, public-private scoping and list filtering
• Act on machines in a configuration as a unit: suspend, multi-snapshot, revert to, shutdown, turn on, turn off, suspend, reset, deploy, undeploy, clone, capture to library, and modify properties
• Share templates and multi-machine configurations between users
• Share captured, live configurations via URL-based “LiveLink” capability
• Interact with all configuration consoles side-by-side on a single browser page
• Setup machines in a configuration to boot in controlled sequence
Configuration Library
• Near-instantaneous check out of configurations with memory and CPU state preserved
• Simultaneous use of library configurations by multiple users without changing MAC and IP addresses or SID, using

VMware network fencing
• Efficient storage compression algorithms maximize library entries per storage server
• Public and private library entries
• Public-private scoping and list filtering of library
Media Library
• Central repository for all development and test media
• Tag media with descriptive attributes
• Upload media from the browser or directly to the file share
• Easily make media available to VMware-managed virtual machines
• Public-private scoping and list filtering of media library
Deployment
• Browser- or Web service-initiated deployment
• Copy from or execute from storage server
• Configuration or machine deployment granularity
• Automatic load balancing of machines deployed on host servers
• Side-by-side execution of cloned configurations across physical server boundaries, even when IP addresses are duplicated on a subnet
• Selectable provisioning heuristic – enabling rapid startup and maximum performance when deploying configurations
Machine Templates
• Instant creation of new machines from templates with distinct personalization-automated assignment of MAC and IP addresses, and SID (for Windows systems)
• New template creation via cloning and modification of existing templates
• Import pre-existing virtual machine images for use as templates
• Automated installation of mouse and keyboard enhancements for virtual machines
• Optional physical-to-virtual (P2V) capability for storage and management of existing machines
• Public-private scoping and list filtering of templates
Resource Management
• Automated tracking and issuance of IP addresses
• Storage server management: add, delete, refresh and modify properties
• Managed server management: add, delete, refresh, agent upgrade, remove from service and modify properties
Monitoring
• Active unified “in motion” view of server pool and virtual machine operations
• Drill-down on server, configuration and machine details
• All deployed machines view
• Comprehensive event and job log viewer
Web Services and Command Line Interface (Automation API)
• Full programmatic control of capture-and-restore operations
• Out-of-the-box automation with leading test automation tools
• Standards-based interface (SOAP, WSDL, HTTP)
• Sample .NET and Java code illustrating use of APIs
Administration and Security
• User, permission and quota management
• Out-of-the-box support for LDAP
• Administrator role assignment to multiple users
• User self-managed preferences
• Context-sensitive on-screen help
Installation
• Standard Windows setup.exe installer for all components
• Automated addition of managed servers to deployment pool
Supported Managed Server Environments
• VMware Infrastructure 3

VMWare relases VMWare Converter 3 Beta Refresh

From the VMWare Converter 3 beta refresh release notes (found at virtualization.info):

VMware Converter provides an easy-to-use, scalable solution for migrations of machines, both physical to virtual and virtual to virtual. Optimized for mass migration, VMware Converter is equally effective for single-machine conversions. With its comprehensive and comprehensible wizards and task manager, VMware Converter imports virtual machines faster, with fewer manual steps required, and fewer source hardware limitations than other methods. Converter can, with its ability to hot clone, import with no downtime on the source physical machine.

VMware Converter combines and expands the functionality available in P2V Assistant and Virtual Machine Importer. It eases interoperability among VMware hosted products (Workstation, VMware Server, and VMware Player), VirtualCenter-managed ESX Server 3.x and 2.5.x, and unmanaged ESX Server 3.x.

Import from Physical Machines
(Source)

  • VMware Converter can hot clone and reconfigure any local or remote physical machine running an operating system noted in Platforms
  • VMware Converter Boot CD can be started from, and clone, local machines outfitted with storage controllers and network adapters that Microsoft lists as supported in Windows 2003

Import from Various Third-Party Formats and VMware Products
(Source)
  • Microsoft Virtual PC (version 7 and higher)
  • Microsoft Virtual Server (any version)
  • Symantec Backup Exec System Recovery images1
  • VMware Workstation 4.x virtual machine (compatible with VMware GSX Server 3.x)
  • VMware Workstation 5.x virtual machine (compatible with VMware Player and VMware Server 1.x)
  • VMware ESX Server 3.x
  • VMware ESX Server 3.x (when managed by VirtualCenter 2.x)
  • VMware ESX Server 2.5.x (when managed by VirtualCenter 2.x)
Export to a Virtual Machine for
VMware Workstation and Datacenter Products
(Destination)
  • VMware Workstation 4.x virtual machine (compatible with VMware GSX Server 3.x, ESX Server 2.5.x)2
  • VMware Workstation 5.x virtual machine (compatible with VMware Player and VMware Server 1.x)3
  • VMware ESX Server 2.5.x (when managed by VirtualCenter 2.x)
  • VMware ESX Server 3.x (when managed by VirtualCenter 2.x)
  • VMware ESX Server 3.x

Not Supported:

  • VMware ESX Server 2.5.x when managed by VirtualCenter 1.x
  • VMware ESX Server 2.5.x unmanaged

1. The Symantec family of products includes the Backup Exec System Recovery (formerly LiveState Recovery) products and the Norton Ghost 9 (and higher) products. Only images from the Backup Exec System Recovery products are fully supported, but images from Norton Ghost 9 (and higher) are likely to work.

2. For ESX Server 2.5.x the .vmdk files must be imported using the vmkfstools utility.

3. Only Workstation 5.5 can power on linked imports of .sv2i images.

For more info please visit VMWare’s Converter 3 Beta Refresh page.