We have a client with about 70 VMs. We're using View 5.1. Last week, they had a bad electrical storm that did some damage to other parts of the network. At some point during the storm, they lost power, power was down overnight, backup batteries eventually ran out, and some of our VM servers didn't shut down gracefully (i.e., they crashed).
Ever since we got them back up, we've been having some strange issues with the VMs and their virtual NICs. Every now and then (roughly a couple times an hour at random), the View Console will suddenly show about half of the VMs as "Agent Unreachable." After a few minutes, they'll all be back to normal. We rebuilt a couple of the pools last night to see if that would resolve anything, but during the rebuild, a couple of the machines showed "Error" and a handful showed "Agent Unreachable" again. We discovered that to resolve this, we had to manually log into each of the affected machines and each "Disable" and "Enable" the virtual NICs (these are WinXP SP3 VMs) or remove the NIC from the VM and re-add it. Removing and re-adding has the effect of then getting a completely new lease since the re-added NIC has a different MAC address, which isn't cool.
I was thinking we'd re-install the Connection server, Composer and vCenter tonight and maybe rebuild all the pools with a fresh snapshot, so I was updating the master template we use for all the VMs (they're linked clones). But I noticed whenever I reboot the master image, it comes up with a self-assigned IP address, and I have to either disable/enable or remove/re-add the NIC.
It may be that there's something else going on on the network, but DHCP is working for all the other physical workstations (about 100 of them), and it's only the virtuals that are having the issue.
I've tried rebooting everything (vCenter server, Connection server, will try rebooting DC tonight). I've also tried giving a static IP to the VMs when they're having the trouble, but they can't ping anywhere except their own address.
Any ideas? We're basically at the point where we start reinstalling everything, which isn't the most elegant solution.
Thanks.