vm

Lately I’ve been scouring the web for used gear because there seems to be an influx of incredibly powerful stuff at amazing prices.

This all came about with me wanting to learn ESX 3.5, and needing the hardware as well as the network to run a feasible set up, with DRS, HA and Storage Vmotion. And I did it – on the cheap.

The first thing that you should know about ESX 3.5 is that it now works with many non/budget-RAID SATA chipsets, though not supported. Two that are readily available are Intel’s ICH5 and Silicone Image’s Sil SATA line. This typically depends on the BIOS you are using, but in regards to the ICH5, you’ll want to disable IDE compatibility mode, and as for Sil – you’ll sometimes want to turn on the RAID (though some Sil single channel cards also work, ie the Vantec SATA 1-port).

The second is that drive space is inexpensive. A 500GB SATAII Western Digital drive will only set you back about $100 CAD/USD. Two of these gives you redundancy! Combine the cheap drives with software like FreeNAS or OpenFiler, and you have yourself a 1TB iSCSI NAS for a fraction of the cost/MB of larger solutions. Just don’t pretend it’s foolproof… With any proper iSCSI, you’ll want some nice and spiffy ethernet cards, and in my case I used the tried-and-true Intel Pro 100 successor, the Pro 1000. You can find the Pro1000 GT for roughly $40, and a PCIe version of similar capabilities for about the same amount. The PCI version is compatible with VMware ESX 3.5, OpenFiler, FreeNAS, and Windows Server 2008. I hear the same goes for the PCIe version, and I will be able to let you know shortly.

Since we’re on the topic of networking, you’re going to either want two gigabit (gbit) switches, or a nice gbit switch with more ports and VLAN ability. I lucked out, and got a used 3Com SuperStack III (3C17706) for next to nothing. I’ve seen plenty of somewhat lesser-known (but just as nice) gigabit HP Procurve, Extreme Networks, and even Dell gear at plain stupid prices on ebay and Craigslist. Seriously. I’m talking 50$ for a 24 port 10/100/1000 switch! The trick on ebay is to not bid at all on stuff until it is about to end… then just pick it up. Well I guess everyone has figured that out by now, but it still works. Don’t draw attention to it by watching it like a hawk – just set up instant messaging reminders, and swoop in. As for Craigslist, I have RSS feeds for things I am interested in: 1U, 2U, 3U, 4U, 6U, rack, rackmount, etc. I check these on a regular basis, and make sure to email right away. Craigslist people are friendly, but will typically sell to the person who a) emails first, b) offers to pick it up the soonest, and c) doesn’t give them a hard time.

Now we’re into routing, mostly because I want to talk about it. This setup does not require any routing at all, but it’s a bit better to have a protected connection to the internet. My personal opinion here is to avoid Cisco at all costs, as recently the re-licensing has made buying one used a lot more expensive than in the past. That said, I do, in fact, own a Cisco router – what can I say! It’s like the gold standard. Of course my opinions are my own, and not that of my employer. For a cheaper routing solution, look to used Juniper, SonicWALL, and even open source stuff like Astaro (which also happens to run in VM…) Peronsally, I run a home licensed Astaro ten user virtual machine, a Juniper 5GT wireless, and barely use the Cisco 2611. That’s just me. If you’re having a hard time finding the Astaro licensing, just let me know, but rest assured it does exists, and is perhaps the EASIEST way to turn on VPN so you can have access to your virtual lab anywhere.

Alright, now we’re at the meat and potatoes – CPU and memory, the power behind all of this. Now, if power isn’t really a big deal, but you want to learn the cool features like the afore-mentioned DRS, HA and Storage Vmotion, the main thing you will want is memory. I’ve found that buying enthusiast RAM on Craigslist is VERY easy. Pick a brand like Crucial, OCZ or Kingston HyperX, and you’re bound to have masses of kids who all read the same articles, and are all selling the same RAM used, pretty much at the same time. Watch the trends, and you can easily build 4 boxes running over 3GB or RAM each, for cheap. Dirt cheap.

If you’re going the consolidation route, your best bet (used) is an Opteron solution. While you can find Opteron 185 and 175 chips on ebay, I find that they are asking a fortune for them because they are socket 939, and are considered top of the line for the specific platform. If you opt for the 165 dual core version, you can use cheap enthusiast RAM with great timings, and get a pretty good processor at the same time. It also means that you can get a motherboard to support both pretty much anywhere at bargain basement cost, and one that will typically have a Sil SATA chipset, or you can add one later.

In the case of the multi-box scenario – I’m using 4 Intel P4 3.0GHz HyperThreading processors. Not powerhouses by any means, but when it comes to storage, you can get Intel P4 motherboards that have ICH5 chipsets very easily because they well so prolific at the time – just make sure to watch those temps.

I think that kinda sums it up, as far as a used, cheap, VMware ESX 3.5 lab goes.

If you have any questions, feel free to shout them out.