Power supply fails, the secondary takes over, the machine keeps running. The dead PSU can be pulled out and a new one hot-plugged (put in, without turning the machine off)
Network port fails, secondary (or, these days, tertiary or quaternary) takes over. You pull the dead network port/adapter and replace it and it starts working again.
Drive in RAID array fails, the rest carry on and the dead drive can be swapped out and the RAID rebuilt.
A catastrophic failure or corrupted system drive, for example, will still cause the system to fail, but a lot of key components are duplicated and hot-swappable, meaning the parts can fail, without the system keeling over.
There, motherboard or memory failure and things like fire and lightning strikes are about the only things to stop it working, other than the OS getting its knickers in a twist.
With redundant, not only do you have the resilience, you also have a whole other system that is shadowed and can take over when the primary fails - usually in a separate fire section and with a separate mains power feed and a separate network. Preferably in another building or on another site, if possible.