This View instance has 4 connection brokers running on Windows Server 2008 R2:
The environment was upgraded 2 years ago from view 4.6 to view 5.0. When View 4.6 was used there were 2 additional Windows 32-bit connection brokers in the mix as well but they were both disabled during the 5.0 upgrade since View 5.0 requires 64-bit Windows. They were disabled but NOT completely removed from View. As part of my troubleshooting of this issue I decided to remove those two disabled connection brokers from the system since it seemed as though they were causing some syncing errors between the brokers.
Here are my steps I’ve taken so far and in this order as far as I can remember them:
1.) Restarted the vCenterServer services including VMware Management webservices
2.) Full reboot the vCenter server
3.) Started digging through the Connection broker logs to see if anything stood out in there – found that there was a WARNING message about not being able to connect to https://<vcenterserver>:443/sdk. Found a number of articles and blog posts about the service account no longer working. Verified that the service account was not locked out and was working by logging onto the vCenter server via RDP successfully.
4.) Edited the vCenter server entry in View Manager and re-typed in the account password and to ensure it was still able to connect successfully and it can.
5.) Further research on VMware KB site and communities found an article stating that the time on the connection brokers may be out of sync. Logged onto each one of the brokers and double-checked the time was all in sync and it is.
6.) Checked to be sure all the LDAP databases were syncing properly – this is where I decided to remove the old un-used brokers from the environment and force a successful replication to each of the brokers neighbors using repadmin utility.
7.) After forcing the replication I went in a restarted the vCenter Mangment web services again. – At this point the connection between the broker and vCenter server came back but the pool that has a pending delete job did not continue and appeared stuck.
8.) After about 5 or 10 minutes the communication error with the vCenter popped back up again.
Also of note, this environment is connected to another vCenter and that one is working fine with no errors or stuck tasks.
Anyone have any additional things I should try?
Thanks,
Jason D.