We had a power failure that took down our EMC VNXe and 3 VMware hosts. When power was restored the VNXe starts up but it takes almost 15 minutes to be "ready" and actively serving up data. The ESXi hosts boot up about 5 minutes. But now we have vSphere booted but no storage. So VMs don't start and my phone starts ringing. I drive to the datacenter, reboot the ESXi hosts and all is well. I've been through this cycle a few times now over the past couple years and it occurs to me now that someone must have figured out how to engineer this better.
Is there a way to get ESXi to recognize that there's no storage and just reboot? Or to refresh itself occasionally so that when storage becomes available everything starts working?
The environment has no other physical machines. So DCs, DNS, DHCP, NTP is all served up by VMs.
We have a UPS but it provides about 30 minutes of power. There's no generator and there's not going to be one so that's not a fix here -- as much as I'd like it to be.
Thanks all!