Forced to Virtualize

I got that phone call that no IT professional wants to get, on Saturday, a user logged in and a critical server is down. Unfortunately, the call wasn't completely unexpected. I had been having problems with this server for a couple weeks and neither the server vendor nor I could pinpoint the problem. At first, the server would reboot during backups, some of the time. Troubleshot the problem and came up with firmware updates for everything related to the storage system. That did not fix the problem. Thinking that it was storage related I moved the file shares from the internal direct attached RAID to our new EqualLogic iSCSI SAN. Problem seemed to go away and it worked well for two weeks (while I was out for our baby's birth, it was a God thing! Praise the Lord!) but this last week it reared its ugly head again. I started getting iSCSI timeout errors during high I/O times and the server service would stop responding.

After I get the call, I VPN into the office and log into the problem server and find that logical disks residing on the SAN are missing and the iSCSI Initiator cannot connect to them. I tinker, restart and tinker some more, but I cannot reinitialize the iSCSI disks. I head into the office and after I cold boot, the iSCSI connections are back online. Since this server is our main campus AD Global Catalog, File and Print, DFS Root, and PKI I cannot afford any downtime during business hours so I decide to virtualize.

I created a new iSCSI target and point the VMWare Converter to the iSCSI target and about 25 minutes later I have a virtual machine. I shutdown the Physical Server and configure and start the new VM on a copy of VMWare Server that I just installed on my beefiest server. After lots of testing I must say that this Virtualization stuff is great! Especially since a physical server can be converted to a virtual server! By the way, this is the first time I've used virtualization in the server room but it really saved the day!

Now I will get this physical server repaired, upgrade the memory and make it a VMWare Host. I can now add redundancy and resiliency to the network via VMs and now adding hardware to the rack. This was a real blessing in disguise.

Comments

[Beats by dr dre headphones]

Why would it take FOUR hours on the phone with Microsoft. Why does it always take so long???