We tried rebooting a new VM we just created from a physical server the other day and it fails when powering on. The event states:
An error was received from the ESX host while powering on VM Peterbuilt.
Failed to start the virtual machine.
Module DiskEarly power on failed.
Cannot open the disk '/vmfs/volumes/vsan:528a8f794664ebd8-6300c6c1887b898b/17ee6253-9171-b202-26f4-842b2b78ebfb/Peterbuilt_1-000002.vmdk' or one of the snapshot disks it depends on.
5 (Input/output error)
The vmware.log from the first time it failed power on contains this:
2014-05-03T10:18:40.564Z| Worker#1| I120: DISK: OPEN scsi0:1 '/vmfs/volumes/vsan:528a8f794664ebd8-6300c6c1887b898b/17ee6253-9171-b202-26f4-842b2b78ebfb/Peterbuilt_1-000002.vmdk' persistent R[]
2014-05-03T10:18:40.564Z| Worker#0| I120: DISK: OPEN scsi0:0 '/vmfs/volumes/vsan:528a8f794664ebd8-6300c6c1887b898b/17ee6253-9171-b202-26f4-842b2b78ebfb/Peterbuilt-000002.vmdk' persistent R[]
2014-05-03T10:18:40.659Z| Worker#0| I120: DISKLIB-VMFS : "vsan://1d846353-60b1-b2e4-7696-d4ae52836f43" : open successful (10) size = 32462929920, hd = 29894442. Type 8
2014-05-03T10:18:40.659Z| Worker#0| I120: DISKLIB-DSCPTR: Opened [0]: "vsan://1d846353-60b1-b2e4-7696-d4ae52836f43" (0xa)
2014-05-03T10:18:40.659Z| Worker#0| I120: DISKLIB-LINK : Opened '/vmfs/volumes/vsan:528a8f794664ebd8-6300c6c1887b898b/17ee6253-9171-b202-26f4-842b2b78ebfb/Peterbuilt-000002.vmdk' (0xa): vmfsSparse, 62912512 sectors / 30 GB.
2014-05-03T10:18:40.671Z| Worker#1| I120: DISKLIB-VMFS : "vsan://1f846353-d29f-90dd-fe1b-d4ae52836f43" : open successful (10) size = 576889700352, hd = 32384813. Type 8
2014-05-03T10:18:40.671Z| Worker#1| I120: DISKLIB-DSCPTR: Opened [0]: "vsan://1f846353-d29f-90dd-fe1b-d4ae52836f43" (0xa)
2014-05-03T10:18:40.671Z| Worker#1| I120: DISKLIB-LINK : Opened '/vmfs/volumes/vsan:528a8f794664ebd8-6300c6c1887b898b/17ee6253-9171-b202-26f4-842b2b78ebfb/Peterbuilt_1-000002.vmdk' (0xa): vmfsSparse, 1118001152 sectors / 533.1 GB.
2014-05-03T10:18:40.683Z| Worker#1| I120: OBJLIB-VSANOBJ: VsanObjReadPolicy: failed to readPolicy for object: Input/output error (327682).
2014-05-03T10:18:40.683Z| Worker#1| I120: OBJLIB-VSANOBJ: VsanObjGetExtParams: Could not read policy for 'vsan://1eee6253-1e80-39fa-229e-842b2b78ebfb'.
2014-05-03T10:18:40.683Z| Worker#1| I120: OBJLIB-LIB: ObjLib_GetExtParams : Failed to get ext params : Input/output error (327684)
2014-05-03T10:18:40.683Z| Worker#1| I120: DISKLIB-VMFS : "vsan://1eee6253-1e80-39fa-229e-842b2b78ebfb" : failed to open (Input/output error): . Type 3
2014-05-03T10:18:40.683Z| Worker#1| I120: DISKLIB-LINK : "/vmfs/volumes/vsan:528a8f794664ebd8-6300c6c1887b898b/17ee6253-9171-b202-26f4-842b2b78ebfb/Peterbuilt_1.vmdk" : failed to open (Input/output error).
2014-05-03T10:18:40.683Z| Worker#1| I120: DISKLIB-CHAIN :"/vmfs/volumes/vsan:528a8f794664ebd8-6300c6c1887b898b/17ee6253-9171-b202-26f4-842b2b78ebfb/Peterbuilt_1-000002.vmdk": Failed to open parent "/vmfs/volumes/vsan:528a8f794664ebd8-6300c6c1887b898b/17ee6253-9171-b202-26f4-842b2b78ebfb/Peterbuilt_1.vmdk": Input/output error.
2014-05-03T10:18:40.683Z| Worker#1| I120: DISKLIB-CHAIN : "/vmfs/volumes/vsan:528a8f794664ebd8-6300c6c1887b898b/17ee6253-9171-b202-26f4-842b2b78ebfb/Peterbuilt_1.vmdk" : failed to open (The parent of this virtual disk could not be opened).
2014-05-03T10:18:40.684Z| Worker#1| I120: DISKLIB-VMFS : "vsan://1f846353-d29f-90dd-fe1b-d4ae52836f43" : closed.
2014-05-03T10:18:40.684Z| Worker#1| I120: DISKLIB-LIB : Failed to open '/vmfs/volumes/vsan:528a8f794664ebd8-6300c6c1887b898b/17ee6253-9171-b202-26f4-842b2b78ebfb/Peterbuilt_1-000002.vmdk' with flags 0xa The parent of this virtual disk could not be opened (23).
2014-05-03T10:18:40.704Z| Worker#0| I120: DISKLIB-VMFS : "vsan://1cee6253-07b9-e212-6f6a-842b2b78ebfb" : open successful (14) size = 32211206144, hd = 32581421. Type 3
2014-05-03T10:18:40.704Z| Worker#0| I120: DISKLIB-DSCPTR: Opened [0]: "vsan://1cee6253-07b9-e212-6f6a-842b2b78ebfb" (0xe)
2014-05-03T10:18:40.704Z| Worker#0| I120: DISKLIB-LINK : Opened '/vmfs/volumes/vsan:528a8f794664ebd8-6300c6c1887b898b/17ee6253-9171-b202-26f4-842b2b78ebfb/Peterbuilt.vmdk' (0xe): vmfs, 62912512 sectors / 30 GB.
2014-05-03T10:18:40.704Z| Worker#0| I120: DISKLIB-CHAINESX : ChainESXOpenSubChain: numLinks = 2, numSubChains = 1
2014-05-03T10:18:40.704Z| Worker#0| I120: DISKLIB-CHAINESX : ChainESXOpenSubChain:(0) fid = 32581421, extentType = 2
2014-05-03T10:18:40.704Z| Worker#0| I120: DISKLIB-CHAINESX : ChainESXOpenSubChain:(1) fid = 29894442, extentType = 0
2014-05-03T10:18:40.705Z| Worker#0| I120: DISKLIB-LIB : Resuming change tracking.
2014-05-03T10:18:40.711Z| Worker#0| I120: DISKLIB-CBT : Initializing ESX kernel change tracking for fid 29894442.
2014-05-03T10:18:40.711Z| Worker#0| I120: DISKLIB-CBT : Successfuly created cbt node 1ca272e-cbt.
2014-05-03T10:18:40.711Z| Worker#0| I120: DISKLIB-CBT : Opening cbt node /vmfs/devices/cbt/1ca272e-cbt
2014-05-03T10:18:40.711Z| Worker#0| I120: DISKLIB-LIB : Opened "/vmfs/volumes/vsan:528a8f794664ebd8-6300c6c1887b898b/17ee6253-9171-b202-26f4-842b2b78ebfb/Peterbuilt-000002.vmdk" (flags 0xa, type vmfs).
2014-05-03T10:18:40.711Z| Worker#0| I120: DISK: Disk '/vmfs/volumes/vsan:528a8f794664ebd8-6300c6c1887b898b/17ee6253-9171-b202-26f4-842b2b78ebfb/Peterbuilt-000002.vmdk' has UUID '60 00 c2 9c 4d 6e 95 99-d1 c1 5f c9 c0 e0 df e3'
2014-05-03T10:18:40.711Z| Worker#0| I120: DISK: OPEN '/vmfs/volumes/vsan:528a8f794664ebd8-6300c6c1887b898b/17ee6253-9171-b202-26f4-842b2b78ebfb/Peterbuilt-000002.vmdk' Geo (3916/255/63) BIOS Geo (3916/255/63)
2014-05-03T10:18:40.717Z| Worker#1| I120: DISKLIB-VMFS : "vsan://1f846353-d29f-90dd-fe1b-d4ae52836f43" : open successful (8) size = 576889700352, hd = 30091058. Type 8
2014-05-03T10:18:40.717Z| Worker#1| I120: DISKLIB-DSCPTR: Opened [0]: "vsan://1f846353-d29f-90dd-fe1b-d4ae52836f43" (0x8)
2014-05-03T10:18:40.717Z| Worker#1| I120: DISKLIB-LINK : Opened '/vmfs/volumes/vsan:528a8f794664ebd8-6300c6c1887b898b/17ee6253-9171-b202-26f4-842b2b78ebfb/Peterbuilt_1-000002.vmdk' (0x8): vmfsSparse, 1118001152 sectors / 533.1 GB.
2014-05-03T10:18:40.717Z| Worker#1| I120: DISKLIB-CHAINESX : ChainESXOpenSubChain: numLinks = 1, numSubChains = 1
2014-05-03T10:18:40.717Z| Worker#1| I120: DISKLIB-CHAINESX : ChainESXOpenSubChain:(0) fid = 30091058, extentType = 0
2014-05-03T10:18:40.718Z| Worker#1| I120: DISKLIB-LIB : Resuming change tracking.
2014-05-03T10:18:40.722Z| Worker#1| I120: DISKLIB-CBT : Initializing ESX kernel change tracking for fid 30091058.
2014-05-03T10:18:40.723Z| Worker#1| I120: DISKLIB-CBT : Successfuly created cbt node 233273a-cbt.
2014-05-03T10:18:40.723Z| Worker#1| I120: DISKLIB-CBT : Opening cbt node /vmfs/devices/cbt/233273a-cbt
2014-05-03T10:18:40.723Z| Worker#1| I120: DISKLIB-LIB : Opened "/vmfs/volumes/vsan:528a8f794664ebd8-6300c6c1887b898b/17ee6253-9171-b202-26f4-842b2b78ebfb/Peterbuilt_1-000002.vmdk" (flags 0x8, type vmfsSparse).
2014-05-03T10:18:40.724Z| Worker#1| I120: OBJLIB-VSANOBJ: VsanObjReadPolicy: failed to readPolicy for object: Input/output error (327682).
2014-05-03T10:18:40.724Z| Worker#1| I120: OBJLIB-VSANOBJ: VsanObjGetExtParams: Could not read policy for 'vsan://1eee6253-1e80-39fa-229e-842b2b78ebfb'.
2014-05-03T10:18:40.724Z| Worker#1| I120: OBJLIB-LIB: ObjLib_GetExtParams : Failed to get ext params : Input/output error (327684)
2014-05-03T10:18:40.724Z| Worker#1| I120: DISKLIB-VMFS : "vsan://1eee6253-1e80-39fa-229e-842b2b78ebfb" : failed to open (Input/output error): . Type 3
2014-05-03T10:18:40.724Z| Worker#1| I120: DISKLIB-LINK : "/vmfs/volumes/vsan:528a8f794664ebd8-6300c6c1887b898b/17ee6253-9171-b202-26f4-842b2b78ebfb/Peterbuilt_1.vmdk" : failed to open (Input/output error).
2014-05-03T10:18:40.724Z| Worker#1| I120: DISKLIB-CHAIN : "/vmfs/volumes/vsan:528a8f794664ebd8-6300c6c1887b898b/17ee6253-9171-b202-26f4-842b2b78ebfb/Peterbuilt_1.vmdk" : failed to open (Input/output error).
2014-05-03T10:18:40.724Z| Worker#1| I120: DISKLIB-LIB : Failed to open '/vmfs/volumes/vsan:528a8f794664ebd8-6300c6c1887b898b/17ee6253-9171-b202-26f4-842b2b78ebfb/Peterbuilt_1.vmdk' with flags 0xe Input/output error (327689).
2014-05-03T10:18:40.724Z| Worker#1| I120: DISK: Cannot open disk "/vmfs/volumes/vsan:528a8f794664ebd8-6300c6c1887b898b/17ee6253-9171-b202-26f4-842b2b78ebfb/Peterbuilt_1.vmdk": Input/output error (327689).
2014-05-03T10:18:40.724Z| Worker#1| I120: DISK: Cannot open disk '/vmfs/volumes/vsan:528a8f794664ebd8-6300c6c1887b898b/17ee6253-9171-b202-26f4-842b2b78ebfb/Peterbuilt_1.vmdk' : Input/output error.
2014-05-03T10:18:40.725Z| Worker#1| I120: DISKLIB-CBT : Shutting down change tracking for untracked fid 30091058.
2014-05-03T10:18:40.725Z| Worker#1| I120: DISKLIB-CBT : Successfully disconnected CBT node.
2014-05-03T10:18:40.737Z| Worker#1| I120: DISKLIB-VMFS : "vsan://1f846353-d29f-90dd-fe1b-d4ae52836f43" : closed.
2014-05-03T10:18:40.742Z| vmx| I120: DISK: Opening disks took 179 ms.
2014-05-03T10:18:40.742Z| vmx| I120: Module DiskEarly power on failed.
2014-05-03T10:18:40.742Z| vmx| I120: VMX_PowerOn: ModuleTable_PowerOn = 0
2014-05-03T10:18:40.742Z| vmx| I120: scsi0:0: numIOs = 0 numMergedIOs = 0 numSplitIOs = 0 ( 0.0%)
2014-05-03T10:18:40.742Z| vmx| I120: Closing disk scsi0:0
2014-05-03T10:18:40.743Z| vmx| I120: DISKLIB-CBT : Shutting down change tracking for untracked fid 29894442.
2014-05-03T10:18:40.743Z| vmx| I120: DISKLIB-CBT : Successfully disconnected CBT node.
2014-05-03T10:18:40.748Z| vmx| I120: DISKLIB-VMFS : "vsan://1d846353-60b1-b2e4-7696-d4ae52836f43" : closed.
2014-05-03T10:18:40.749Z| vmx| I120: DISKLIB-VMFS : "vsan://1cee6253-07b9-e212-6f6a-842b2b78ebfb" : closed.