Folk have phones that can play these games, because they buy the phones anyway. Then the games are one or two taps away in the respective App store/Market.
I don't believe this market will die off any time soon, but I believe the novelty will wear off and the numbers being sold will decline.
As for any declines in Nintendo/PSP game sales, I doubt mobile games are the main cause. I haven't bought a 3DS because there aren't really any games for it that appeal to me, I'm blind in one eye (making the key feature worthless to me), and the battery life isn't good enough. I'm sure two of those three things have prevented a lot of people buying 3ds games, as well as general hard times.