Here is a sample output of esxtop on a host running no VMs at all.
9:58:51am up 61 days 13:51, 597 worlds, 0 VMs, 0 vCPUs; CPU load average: 0.08, 0.09, 0.11 PCPU USED(%): 61 0.2 0.9 0.2 0.4 0.3 0.3 0.1 AVG: 8.0 PCPU UTIL(%): 83 0.3 1.3 0.3 0.3 0.4 0.2 0.1 AVG: 10 CORE UTIL(%): 83 1.6 0.7 0.3 AVG: 21 ID GID NAME NWLD %USED %RUN %SYS %WAIT %VMWAIT %RDY %IDLE %OVRLP %CSTP % 1 1 system 208 45.28 783.63 0.00 19653.99 - 39.23 0.00 13.77 0.00 14004098 14004098 esxtop.4079612 1 3.74 2.84 0.02 95.57 - 0.00 0.00 0.01 0.00 14002722 14002722 sh.4079405 1 0.07 0.06 0.00 98.36 - 0.00 0.00 0.00 0.00 14003090 14003090 python.4079451 29 0.05 0.06 0.00 2854.05 - 0.01 0.00 0.00 0.00 6062 6062 hostd.2098611 31 0.03 0.04 0.00 3050.82 - 0.01 0.00 0.00 0.00 7488 7488 net-lbt.2098818 1 0.03 0.04 0.00 98.38 - 0.00 0.00 0.00 0.00 9880 9880 vpxa.2099137 38 0.03 0.02 0.00 3739.81 - 0.01 0.00 0.00 0.00 5555 5555 hostdCgiServer. 12 0.01 0.02 0.00 1180.95 - 0.01 0.00 0.00 0.00 2198 2198 net-lacp.209774 3 0.01 0.01 0.00 295.24 - 0.01 0.00 0.00 0.00 14002354 14002354 sshd.4079350 1 0.01 0.01 0.00 98.41 - 0.00 0.00 0.00 0.00 6231 6231 rhttpproxy.2098 27 0.01 0.00 0.00 2657.21 - 0.00 0.00 0.00 0.00 8 8 helper 154 0.00 0.00 0.00 15153.51 - 0.00 0.00 0.00 0.00 2320 2320 vmkiscsid.20977 2 0.00 0.01 0.00 196.83 - 0.00 0.00 0.00 0.00 5178 5178 vmware-usbarbit 1 0.00 0.00 0.00 98.41 - 0.00 0.00 0.00 0.00
You see a pretty reasonably high %USED for the system world, and it doesn't seem to be enough to troubleshoot.
So what I do next is use e key to enter the system world.
This is what I get :
10:01:00am up 61 days 13:53, 597 worlds, 0 VMs, 0 vCPUs; CPU load average: 0.08, 0.09, 0.10 PCPU USED(%): 2.0 1.7 8.1 5.5 3.3 0.7 0.5 0.1 AVG: 2.7 PCPU UTIL(%): 3.5 2.6 11 7.0 2.7 0.8 0.9 0.1 AVG: 3.6 CORE UTIL(%): 6.0 18 3.5 1.0 AVG: 7.1 ID GID NAME NWLD %USED %RUN %SYS %WAIT %VMWAIT %RDY %IDLE %OVRLP %CSTP % 2097153 1 idle1 1 0.00 97.28 0.00 0.00 - 1.12 0.00 0.39 0.00 2097154 1 idle2 1 0.00 93.76 0.00 0.00 - 4.65 0.00 1.87 0.00 2097155 1 idle3 1 0.00 95.47 0.00 0.00 - 2.94 0.00 1.43 0.00 2097156 1 idle4 1 0.00 95.89 0.00 0.00 - 2.52 0.00 0.02 0.00 2097157 1 idle5 1 0.00 97.75 0.00 0.00 - 0.65 0.00 0.03 0.00 2097158 1 idle6 1 0.00 97.83 0.00 0.00 - 0.57 0.00 0.03 0.00 2097159 1 idle7 1 0.00 98.30 0.00 0.00 - 0.10 0.00 0.00 0.00 2097160 1 vmkEventAsyncMs 1 0.00 0.00 0.00 98.40 - 0.00 0.00 0.00 0.00 2097161 1 fastslab 1 0.00 0.00 0.00 98.39 - 0.01 0.00 0.00 0.00 2097162 1 SVGAConsole 1 0.00 0.00 0.00 98.39 - 0.01 0.00 0.00 0.00 2097163 1 debugtermlivedu 1 0.00 0.00 0.00 98.40 - 0.00 0.00 0.00 0.00 2097164 1 logSysAlert 1 0.00 0.00 0.00 98.40 - 0.00 0.00 0.00 0.00 2097165 1 serialLogger 1 0.00 0.00 0.00 98.40 - 0.00 0.00 0.00 0.00 2097166 1 tlbflushcount 1 0.00 0.00 0.00 98.40 - 0.00 0.00 0.00 0.00
Not really useful, so I'm gonna sort by %USED using U key :
10:01:55am up 61 days 13:54, 566 worlds, 0 VMs, 0 vCPUs; CPU load average: 0.08, 0.09, 0.10 PCPU USED(%): 0.3 0.1 5.1 0.2 0.0 53 0.5 0.1 AVG: 7.5 PCPU UTIL(%): 0.4 0.1 4.2 0.3 0.0 75 0.6 0.1 AVG: 10 CORE UTIL(%): 0.5 4.4 75 0.1 AVG: 20 Sort by %used ID GID NAME NWLD %USED %RUN %SYS %WAIT %VMWAIT %RDY %IDLE %OVRLP %CSTP % 2097153 1 idle1 1 0.00 100.84 0.00 0.00 - 0.08 0.00 0.02 0.00 2097154 1 idle2 1 0.00 96.73 0.00 0.00 - 4.18 0.00 0.00 0.00 2097155 1 idle3 1 0.00 100.77 0.00 0.00 - 0.14 0.00 0.01 0.00 2097156 1 idle4 1 0.00 100.88 0.00 0.00 - 0.04 0.00 0.00 0.00 2097157 1 idle5 1 0.00 70.16 0.00 0.00 - 30.75 0.00 12.81 0.00 2097158 1 idle6 1 0.00 100.51 0.00 0.00 - 0.40 0.00 0.07 0.00 2097159 1 idle7 1 0.00 100.84 0.00 0.00 - 0.07 0.00 0.00 0.00 2097160 1 vmkEventAsyncMs 1 0.00 0.00 0.00 100.00 - 0.00 0.00 0.00 0.00 2097161 1 fastslab 1 0.00 0.00 0.00 100.00 - 0.01 0.00 0.00 0.00 2097162 1 SVGAConsole 1 0.00 0.00 0.00 100.00 - 0.01 0.00 0.00 0.00 2097163 1 debugtermlivedu 1 0.00 0.00 0.00 100.00 - 0.00 0.00 0.00 0.00 2097164 1 logSysAlert 1 0.00 0.00 0.00 100.00 - 0.00 0.00 0.00 0.00 2097165 1 serialLogger 1 0.00 0.00 0.00 100.00 - 0.00 0.00 0.00 0.00 2097166 1 tlbflushcount 1 0.00 0.00 0.00 100.00 - 0.00 0.00 0.00 0.00
As you can see, it doesn't really work. You might think that's because pretty much everything is at zero, but if I manage to display much more lines in my terminal output by reducing character size, I find some stuff way down :
ID GID NAME NWLD %USED %RUN %SYS %WAIT %VMWAIT %RDY %IDLE %OVRLP %CSTP % [...] 2097277 1 CpuSchedRealloc 1 0.07 0.06 0.00 98.32 - 0.00 0.00 0.00 0.00 0.00 0.00 2097278 1 CpuMetricsLoadH 1 0.01 0.01 0.00 98.37 - 0.00 0.00 0.00 0.00 0.00 0.00 2097279 1 CpuSchedExtende 1 0.00 0.00 0.00 98.37 - 0.01 0.00 0.00 0.00 0.00 0.00 2097280 1 PktSlabMemorySt 1 0.00 0.00 0.00 98.37 - 0.01 0.00 0.00 0.00 0.00 0.00 2097305 1 DCFlushCaches 1 0.00 0.00 0.00 98.38 - 0.00 0.00 0.00 0.00 0.00 0.00 2097306 1 OCFlush 1 50.76 36.98 0.00 61.40 - 0.00 0.00 0.00 0.00 0.00 0.00 2097341 1 bcflushd 1 0.00 0.00 0.00 98.38 - 0.00 0.00 0.00 0.00 0.00 0.00 2097358 1 VSCSIPoll 1 0.06 0.06 0.00 98.31 - 0.02 0.00 0.00 0.00 0.00 0.00 2097369 1 Storage-APD 1 0.00 0.00 0.00 98.38 - 0.00 0.00 0.00 0.00 0.00 0.00 2097414 1 serialSwitcher 1 0.00 0.00 0.00 98.38 - 0.00 0.00 0.00 0.00 0.00 0.00 2097415 1 logterm 1 0.00 0.00 0.00 98.38 - 0.00 0.00 0.00 0.00 0.00 0.00 2097416 1 logterm-scroll 1 0.00 0.00 0.00 98.38 - 0.00 0.00 0.00 0.00 0.00 0.00 2097417 1 memMap-adj 1 0.00 0.00 0.00 98.38 - 0.00 0.00 0.00 0.00 0.00 0.00 2097418 1 pmemArs 1 0.00 0.00 0.00 98.38 - 0.00 0.00 0.00 0.00 0.00 0.00 2097423 1 NRandomHwrng 1 0.00 0.00 0.00 98.38 - 0.00 0.00 0.00 0.00 0.00 0.00 [...]
So I have two questions :
- Why can't we sort when inside a world in esxtop ?
- What is OCFlush and why it would be taking so much CPU while no VMs are running ?
Why does it matter? This is actually a home lab, and ESX is running on a spare MacBook Pro, bottomline is the fans are making too much noise and I want to fix this!
ESX 6.7.0 (Build 8169922)