ZFS Build Checklist

I’ve decided to replace the Windows Home Server Vail server with something capable of handling newer builds of ZFS and the inherent deduploication.

Here’s a quick kit list and build diary I’ll try to keep up-to-date as I go along.

Kit:

  • Dell Perc6i – this is essentially a port multiplier. I scored it from eBay on the cheap, though it was delivered from Israel, took awhile, and had neither cables nor mounting bracket.
  • OCZ RevoDrive 120GB – Though the RAID controller on this card is not supported in Linux/Solaris, the drives show up as two separate devices as long as you make sure to put it in the right PCIe slot. That means it’s perfect for both ZIL (log) and L2ARC (cache).
  • 2x Intel 80GB X25-M SSDs – these will house the virtual machine files to be deduped. Very reliable drives, and though they might not be the fastest in terms of writes, the speeds are relatively constant which is quite handy compared to solutions that attempt compression like SandForce controllers. ZFS will take care of that, thanks.
  • (IN TRANSIT) 2x Dual Port 1gbit Intel PCIe NICs – I’ll use these for the direct connection to the virtual machine host. Currently one link is used, but when reading from the SSD drives the line is saturated.
  • (IN TRANSIT) 32 Pin SAS Controller To 4x SATA HDD Serial Cable Cord – This is needed to plug in 8 drives to the LSI controller.
  • 5x 1.5TB Seagate hard drives – These will be the bread-and-butter storage running in RAID-Z2 (similar to RAID 6).
  • 3x 3TB Seagate hard drives – These might simply be a large headache, but the plan was to have an extra 3TB RAID-Z2 for backups in another machine. Unfortunately there seem to be issues with drives that are 4k presenting themselves as 512b. I may be able to get around this by hacking or waiting as they become more popular. For now 2 of them are in software RAID1 on a Windows 7 host, and the other remains in the external USB 3 case and is used as a backup drive.
  • NetGear GS108T Switch – A cheap VLAN-capable switch should I decide to use more than 2 bonded ports (I doubt it), currently running the lab.

Macbook Pro 15″ i5 Unibody Upgrades

I’ve finally purchased a MacBook Pro, and things are going pretty well. Most of my work these days involves using servers for heavy lifting, but I still use Windows 7 from time to time, and Lightroom 3 almost all the time.

Unfortunately, Lightroom 3’s catalog is essentially a database of photos, and the more you put in it, the more slowly it will run. In this case, the MacBook Pro’s stock 320GB 5400 RPM hard drive just isn’t cutting the mustard. Simple actions like scrolling through images from the last import can be painful. Using Firefox or Chrome while importing makes everything crawl, and I’m forced to look for entertainment in Meat Space. The horror!

I know, “it can’t be that bad” is what you’re thinking. It is. Imports can take up to an hour. While on vacation, the last thing I want to be doing is waiting for imports of photos I’ve already taken while I could be out taking more photos.

I mentioned the fact that I use Windows 7 on the MBP. This is via either Boot Camp or VMware Fusion (running the Boot Camp partition). Things work swimmingly in Boot Camp, but I really have to be careful in Fusion because many of the newer Mac applications are RAM-hungry, and you start paging to disk quickly. Since the disk is so slow, you’re at a standstill within minutes.

So the problem essentially boils down to two things, both of which could have been resolved at time of purchase had I looked into the specs a bit further.

  1. Not enough memory
  2. Hard drive too slow

Costs add up

The memory upgrade, direct from Apple, via their online store, is a whopping $420. The hard drive upgrade from 320GB 5400 RPM to 500GB 7200 RPM is $158. Together I would have shelled out $578 in order to get the system where I think it needs to be.

Enter the Apple Technician

In the not-so-distant past, I repaired Apple laptops for a certified depot. It used to be pretty difficult as some of the Mac laptops had an inordinate amount of screws of varying sizes and dizzying teardown diagrams. I would say I was competent, but it really wasn’t something enjoyable. That said, I have been out of the game for a bit, and things have seemingly gotten much easier for the majority of Apple laptops. Often, you can simply remove the bottom case to gain access to wireless cards, Bluetooth, SuperDrive, hard drive and memory. And such is the case with the Macbook Pro 15″ i5.

Using the diagrams found at iFixIt, I was able to confirm that only a little bit of work would be needed to perform the upgrades. That means I save money on labour, which isn’t cheap.

Price Comparison

I was able to source hard drives at the 500GB capacity ranger running at 7200RPM for very cheap. I’d be looking at around $80, worst case. But being spoiled on other computers running solid state drives, I thought I should look into the option of adding an SSD instead. Though they have come down in price, getting larger capacity SSD drives can run upwards of $400 easily. Ouch. I decided to settle on one of Seagate’s newly-released “hybrid” drives that combine 4GB of superfast SSD with 500GB of traditional rotating platter storage. This should hopefully give me the best of both worlds. The cost? About $140. That’s definitely a few dollars less than the “off the shelf” Apple price, though it’s also double the cost of a typical 500GB 2.5″ hard disk. But speed is the issue to address, and I’m confident the HDD will address that. My only concern will be the speed of the platters may produce noise.

The memory for a MacBook Pro i5 is slightly harder to find. It took some poking around to find the exact speed and latency of the chips, as I want to make sure the logic board won’t complain, and no unforeseen issues would be introduced. After looking at Kingston’s website, I was able to deduce that the full specifications of the RAM are as follows:

  • Format – 204 pin SODIMM
  • Speed – PC3-8500 / DDR3 1066MHz
  • Latency – 7-7-7-20

This is not cheap memory. We’re talking high speed, high density, low latency RAM. After searching high and low, I came across some Mushkin RAM that was Mac certified. I wasn’t even aware that Mushkin made Mac certified RAM, but boy was I happy. The cost for an 8GB pair of 4GB SODIMM modules was only $260! In case you’re interested, the part number is “996644”, and I still don’t see a better deal from ANY vendor for memory this fast with timings this tight. Even for PC.

Our current total is sitting at $400. That’s less than even the RAM would cost from Apple.

Going Forward

Not to miss any opportunities, I decided to go one step further. Removing the memory and hard drive would leave me with spare parts. These could be sold on Craigslist locally for cheap, or I could re-use them. Use for the hard drive is pretty easy: Time Machine backup. A $20 external AcomData 2.5″ Ruggedized Samurai enclosure would fit the bill well, but the last thing you want to do on vacation is lug around cables and accessories. In my experience, they either get lost or forgotten (or both). This may not be the case for everyone, but I actually rarely use optical media. My data is transferred using USB sticks if I need to sneakernet, over wifi or LAN if I need to backup (and again to another location off-site to be safe) and when I do make audio “mixtapes”, it’s not often as I use an iPod for music.

So here I have a useless device taking up space in the laptop. Some digging, and looking at the tear-down told me a 2.5″ hard drive could fit in there easily. Excellent, a use for the old drive that takes up no extra space! Of course, like many good ideas I think I have come up with first, someone had “been there, done that” before, and you can buy full kits online for cheap. I found two companies that sell these: MCE and OWC. I opted for OWC because I really don’t have a need for the external optical drive that MCE throws in for “free”, creating a $20 difference in price as I have a Lacie DVD-RW already. Cost of this part: $80. (MCE’s is around $100 if you still might need that SuperDrive)

The total now sits at $480. More than the cost of the RAM, but still considerably less than the over $700 cost to have Apple do this at time of purchase. If you had messed up and bought the lower-end 15″ i5 Macbook Pro, there would also be at least an hour of labour on top. Typically that would run about $150.

I’m left with 2x 2GB DDR3 SODIMM modules, which might be hard to get rid of at any price, though they make a good upgrade for Mac Mini users. I’ve looked high and low for DDR3 SODIMM “RAMDisks” to no avail. I realize these aren’t the best devices, and never really had a following, but it would certainly be handy to have on one of the servers. One can only dream, I suppose.

So there you have it, cheap upgrade, easy install, no regrets. Preliminary testing tells me that the boot time has been halved, and Lightroom is much faster, though it’s not as fast as running it on my Mac Pro with SSD.

At some point I will probably look at replacing the second internal drive with a solid state boot drive when I replace the Intel X25-M G2 80GB in the Mac Pro with a SandForce SSD, and I will make sure to post some speeds when that frabjous day finally arrives.

Windows Home Server in VMware Fusion 3

I set off on a quest to get the home backup / media server / remote access solution Windows Home Server with Power Pack 3 running inside of VMware Fusion 3 running on top of Apple OSX Snow Leopard (10.6).

Why, you ask? Simply because I thought I could… and it works good with traderush so please don’t ask me if is traderush legit A little while after downloading the Windows Home Server trial, it became apparent that there was no selection for this operating system. No matter, I thought, it’s based on Windows Server 2003, so I should simply be able to select that, right? Unfortunately not that easy. First, the hard disk type selected by default by VMware Fusion is SCSI. Without a driver disk (virtual floppy), you’ll have no luck. Also, the amount of memory available doesn’t meet the Windows Home Server requirements.

My method?

Try these settings:

– Windows Server 2003 Web Server

– No “easy install” settings

– 512MB RAM

– Remove the default HDD

– Add an 80GB IDE HDD

– Make sure the ISO is mounted

Things seem to be working at this point.

Hope this helps someone, I trawled Google and the Fusion forums with no luck.

g4u and Virtualization

A quick note about g4u – g4u (Ghost for you) is a hard disk imaging tool similar to Norton Ghost, but without the cost. g4u allows you to image hard disks to both files and physical media. This can be accomplished within a physical machine, a virtual machine, or by using an FTP server on either of the two.

Leopard Server on Leopard with VMware Fusion 2.0!

VMware has just announced support for their 61st OS supported by Fusion 2.0 – Mac OS X Server 10.5 (Leopard). This is great news for those looking to test things like the new Active Directory wizards, calendar server and enterprise blogging that come with the new version of the server. Not to mention that because it’s supported by Fusion 2.0, you can do it on your laptop.

Check out the full blog post on Fusion 2.0 Leopard Server support at the VMware VMTN blog site here.

VMware Announces VMSafe Hypervisor Security Platform

As described in the VMware announcement:

[A] new security technology called VMware VMsafe™ that protects applications running in virtual machines in ways previously not possible in physical environments. The VMsafe APIs allow vendors to develop advanced security products that combat the latest generation of malware. VMsafe technology integrates into the VMware hypervisor and provides the transparency to prevent threats and attacks such as viruses, trojans and keyloggers from ever reaching a virtual machine. Twenty security vendors have embraced VMsafe technology and are building products that will further enhance the security of virtual machines, making the virtual environment unmatched in the level of security and protection it provides compared to physical systems.

“VMware already has the most trusted virtualization platform for running applications, and we are now raising the bar on security in ways that physical systems simply cannot match,” said Raghu Raghuram, vice president of datacenter products and solutions. “The industry has come out in full force to support VMware VMsafe technology with plans for a whole new class of security products that offer customers new advantages to running applications in virtual machines.”

So it looks like VMsafe will be a method that security vendors can use in order to ensure that operations in the virtual world will remain just as protected as our meatspace servers

In Dianne Green’s keynote today, you’ll note that almost all of the typical security players have signed up to be part of this platform – I’m guessing it could be quite lucrative, with TCOs everywhere looking for bigger, better tinfoil hats.

Download Thinstall from VMware – Project North Star

VMware has recently updated the VMware Thinstall page and is now offering a download of what they call “Project North Star” – a Thinstall product.

Also not that there is a thinstalled version of Firefox available for download from the same page.

Through Dianne Green didn’t speak a whole lot about Thinstall at VMworld today, it’s good to see VMware at least touch on it a bit as the appvirt vendors are keen to find out what sort of developments are to come.

Novell to Acquire Platespin for 205M

More purchases and conglomeration on the way this year.

The buzz at VMworld Europe, before it has even started, is that Novell is to purchase Canadian data center management software company Platespin. Novell has now posted the news on their site, and many are scrambling to figure out where this will lead them as the software is very popular in the VM market. (UPDATE: Platespin have posted their announcement regarding the Novell acquisition as well)
Novell is starting to grow again: it now owns SuSE Linux, Ximian (original makers of Gnome), and just recently purchased open colaboration vendor SiteScape.

If you’d like more info about the acquisition, feel free to register for the Novell-Platespin webinar.

From the Platespin announcement:

On February 25, 2008, Novell announced that it has entered into a definitive agreement to acquire PlateSpin Ltd. The combination of Novell’s platform and automation management with PlateSpin’s leading solutions for workload relocation, protection and provisioning will give customers the agility to cross physical and virtual boundaries so IT can work together. Both organizations are focused on helping customers maximize the strategic value of the heterogeneous data center. Novell and PlateSpin will deliver products for complete workload lifecycle management and optimization for Linux, UNIX, and Windows operating systems in the physical and virtual data center. Below you will find details about the acquisition and what it means to current and future PlateSpin customers and partners.

From the Novell announcement:

The acquisition of PlateSpin will allow Novell to offer customers a full solution stack with a powerful virtualization platform and a best-in-class heterogeneous management solution. Together, Novell and PlateSpin will solve many of the data center challenges that customers face today, including:

  • Relocation: PlateSpin provides a completely integrated product suite that automates the assessment and migration phases of data center initiatives, like server consolidation, data center relocation and hardware upgrades, to help customers reduce costs, power consumption and space in the data center.
  • Protection: PlateSpin’s disaster recovery solutions offer affordable workload protection that leverages virtualization technology to protect both physical and virtual servers in the data center, for improved security and business continuity.
  • Provisioning: Using PlateSpin’s technologies, customers will have a single approach to imaging and configuring physical and virtual workloads regardless of platform. This eliminates the manual install process and dramatically reduces the time to provision new server workloads. It will also enable customers to address changing resource requirements at peak demand times as well as in test lab scenarios.
  • Optimization and Management: Novell and PlateSpin optimize the balance between physical and virtual infrastructure by automatically monitoring and making infrastructure adjustments based on server availability and workload demand. By automating the process and increasing the visibility into how workloads use physical and virtual resources over time, customers will be able to increase server utilization and optimize their data centers by better addressing common workload movement challenges.

Continue reading

Learning VMware ESX 3.5 on the Cheap

Lately I’ve been scouring the web for used gear because there seems to be an influx of incredibly powerful stuff at amazing prices.

This all came about with me wanting to learn ESX 3.5 and how to build profitable online business, and needing the hardware as well as the network to run a feasible set up, with DRS, HA and Storage Vmotion. And I did it – on the cheap.

The first thing that you should know about ESX 3.5 is that it now works with many non/budget-RAID SATA chipsets, though not supported. Two that are readily available are Intel’s ICH5 and Silicone Image’s Sil SATA line. This typically depends on the BIOS you are using, but in regards to the ICH5, you’ll want to disable IDE compatibility mode, and as for Sil – you’ll sometimes want to turn on the RAID (though some Sil single channel cards also work, ie the Vantec SATA 1-port).

The second is that drive space is inexpensive. A 500GB SATAII Western Digital drive will only set you back about $100 CAD/USD. Two of these gives you redundancy! Combine the cheap drives with software like FreeNAS or OpenFiler, and you have yourself a 1TB iSCSI NAS for a fraction of the cost/MB of larger solutions. Just don’t pretend it’s foolproof… With any proper iSCSI, you’ll want some nice and spiffy ethernet cards, and in my case I used the tried-and-true Intel Pro 100 successor, the Pro 1000. You can find the Pro1000 GT for roughly $40, and a PCIe version of similar capabilities for about the same amount. The PCI version is compatible with VMware ESX 3.5, OpenFiler, FreeNAS, and Windows Server 2008. I hear the same goes for the PCIe version, and I will be able to let you know shortly.

Since we’re on the topic of networking, you’re going to either want two gigabit (gbit) switches, or a nice gbit switch with more ports and VLAN ability. I lucked out, and got a used 3Com SuperStack III (3C17706) for next to nothing. I’ve seen plenty of somewhat lesser-known (but just as nice) gigabit HP Procurve, Extreme Networks, and even Dell gear at plain stupid prices on ebay and Craigslist. Seriously. I’m talking 50$ for a 24 port 10/100/1000 switch! The trick on ebay is to not bid at all on stuff until it is about to end… then just pick it up. Well I guess everyone has figured that out by now, but it still works. Don’t draw attention to it by watching it like a hawk – just set up instant messaging reminders, and swoop in. As for Craigslist, I have RSS feeds for things I am interested in: 1U, 2U, 3U, 4U, 6U, rack, rackmount, etc. I check these on a regular basis, and make sure to email right away. Craigslist people are friendly, but will typically sell to the person who a) emails first, b) offers to pick it up the soonest, and c) doesn’t give them a hard time.

Now we’re into routing, mostly because I want to talk about it. This setup does not require any routing at all, but it’s a bit better to have a protected connection to the internet. My personal opinion here is to avoid Cisco at all costs, as recently the re-licensing has made buying one used a lot more expensive than in the past. That said, I do, in fact, own a Cisco router – what can I say! It’s like the gold standard. Of course my opinions are my own, and not that of my employer. For a cheaper routing solution, look to used Juniper, SonicWALL, and even open source stuff like Astaro (which also happens to run in VM…) Peronsally, I run a home licensed Astaro ten user virtual machine, a Juniper 5GT wireless, and barely use the Cisco 2611. That’s just me. If you’re having a hard time finding the Astaro licensing, just let me know, but rest assured it does exists, and is perhaps the EASIEST way to turn on VPN so you can have access to your virtual lab anywhere.

Alright, now we’re at the meat and potatoes – CPU and memory, the power behind all of this. Now, if power isn’t really a big deal, but you want to learn the cool features like the afore-mentioned DRS, HA and Storage Vmotion, the main thing you will want is memory. I’ve found that buying enthusiast RAM on Craigslist is VERY easy. Pick a brand like Crucial, OCZ or Kingston HyperX, and you’re bound to have masses of kids who all read the same articles, and are all selling the same RAM used, pretty much at the same time. Watch the trends, and you can easily build 4 boxes running over 3GB or RAM each, for cheap. Dirt cheap.

If you’re going the consolidation route, your best bet (used) is an Opteron solution. While you can find Opteron 185 and 175 chips on ebay, I find that they are asking a fortune for them because they are socket 939, and are considered top of the line for the specific platform. If you opt for the 165 dual core version, you can use cheap enthusiast RAM with great timings, and get a pretty good processor at the same time. It also means that you can get a motherboard to support both pretty much anywhere at bargain basement cost, and one that will typically have a Sil SATA chipset, or you can add one later.

In the case of the multi-box scenario – I’m using 4 Intel P4 3.0GHz HyperThreading processors. Not powerhouses by any means, but when it comes to storage, you can get Intel P4 motherboards that have ICH5 chipsets very easily because they well so prolific at the time – just make sure to watch those temps.

I think that kinda sums it up, as far as a used, cheap, VMware ESX 3.5 lab goes.

If you have any questions, feel free to shout them out.

Virtual Desktop Infrastructure 2.0 Download

VMware VDI 2 can now be downloaded from the main VMware site, as of today!

Virtual Desktop Infrastructure allows you to provision desktops as if they were terminal server sessions, but gives users access to their very own desktop.

From VMware’s page on VDM / VDI:

“VMware Virtual Desktop Infrastructure (VDI) is an end-to-end solution for virtual desktop management that gives you the ability to deliver desktops from the data center for greater control and flexibility. With VMware VDI, you get:

  • VMware Infrastructure 3, the proven virtual infrastructure software suite, including VMware VirtualCenter for management
  • VMware Virtual Desktop Manager (VDM) 2, the enterprise-class connection broker that connects remote users to centralized desktops and manages VDI environments”