The Ship of Theseus comes to mind here. Sort of.
If you could swap out all the modules and replace them when they fail at which point would you have been better off just buying a new phone?
It'd also be a development nightmare. The manufacturer of the modules will inevitably bring out newer revisions of their modules.
This gives you even more possible permutations than the current Android phone ecosystem.
Lastly when you've eventually swapped out all the modules for new modules how would the warranty work? At which point is it no longer considered the original phone?
A warranty on each module would be absurdly hard to manage and comprehend for average folk.
"Ah yes sir, but you plugged in the v2.0 camera module to the v1.2 control module, the warranty on the v2.0 module doesnt cover this"
This opens the door to potentially unnecessary upsell.
"Ah yes sir, you'll also need the v2.0 control module for that camera and a v2.0 touch module"
This is A confusing for average consumers, B annoying to develop software for, and C a great way for consumers to be silently ripped off without knowing it.
Modular phones are not the answer I'm afraid. As cool as they would be.
Also, if my little lad got hold of a modular phone I'd likely be walking around with some modules missing quite frequently.
Toddlers are shockingly efficient at dismantling and hiding things.