If done right nothing will go wrong. It is the "something you have" to go along with "something you know". So "done right" would mean not using finger/face ID as a password, but require typing in a password - at least for really important stuff like transferring money out of your investment account.
You've only listed two of the 3 security factors:
1) something you have (e.g. RFID card, token generator, key)
2) something you know (password - not biometrics)
3) something you are (biometrics)
Therefore having to use biometrics1 to unlock the phone would cover that third factor, something you are. Note that something you are is not a password, it is more akin to a username.
So, assuming you have an accurate and reliable biometric system to unlock the phone, you could have a process as follows that covers all three security factors:
1) unlock phone with biometrics
2) retrieve TOTP token from phone app (not an SMS message2)
3) Enter the TOTP token plus your password.
Some of this can be sorta automated, in that rather than manually having to enter the TOTP token, it could be retrieved via NFC or similar technologies and automatically input, it becomes more like a RFID card in this case that is only activated once you have unlocked the phone (or other device) with your biometrics. But you still need to enter a password as the something you know.
Of course, will this ever work in practice in a secure, reliable, easy, cheap and convenient mass-market consumer-friendly fashion? Not in my lifetime I think.
-----------
1 fingerprint, facial recognition, iris scan, dna sample, etc.
2 SMS doesn't satisfy something you have, as you (or an attacker) doesn't need physical access to your phone to receive an SMS message, you just need the phone number, and that can be moved or cloned/duplicated between physical (or virtual) handsets without you knowing, thus defeating the something you have test.