Reply to post:

"They still wait on I/O like any other CPU"

If an HPC application spends much time waiting on IO then someone needs to call in a real HPC expert to give the setup a once-over, because that's a total waste of time (as you rightly point out).