
Re: CPU cycles
Because on an HPC system you have one process to one core as a rule. Typically you will have somewhere between 85-90% of your cores busy at any one time. The slack is mostly down to differing job sizes, and when someone submits a large job you have to hold nodes in reserve waiting for enough to become available to schedule the large job.
Now sure there are some background processes but lets say you have 40 cores in a node then a load average more than 45, say 50 tops will produce an alert because something is wrong. As a sys admin you would be investigating pretty dam sharpish if all or even a significantly number of your nodes where showing a sustained load average above the number of cores on the node.