While I agree that the extra level of confidence supplied by the bit for bit comparison in hash matches is good for peace of mind, whether it is "stupid" to do otherwise depends on the expected collision rate.
Go read up the specs on your favorite hard drives and look at the specs for error rates. Not the ones for corrected or detected errors, but the ones for undetected read and write errors. Yes, those errors are extremely unlikely but if the hash collision was even more unlikely would you still feel you need to do the bit by bit comparison? What's the point, if the underlying hard drives were 1000x more likely to have an undetected error that gave you back false data? Heck, even ECC RAM has an undetected read/write error rate.