The difference between a CEO and a COO is that the CEO will lean more heavily into strategy and external relations, while the COO will lean more heavily into tactics and, well, operations.
What I read into this strange "CEO of a division but not fully CEO, instead of COO" is either the workload has become too much for Nadella alone, or, they are preparing some sort of succesion plan.
As for the AI investments of Microsoft/Nadella "coming to roost" what I can say is that their direct competitors also jumped head first in this fad, so, once the fad fades, comparatively speaking, microsoft will not be worse off compared to the competition. What could REALLY hurt microsoft is that AI ends up NOT being a fad, and Micrsoft NOT being one of the top companies in the space.
Finally, Nadella does not need technical chops to steer the technical side of the bussiness. What he will do is decide which tech initiatives will NOT go forward, which will do, and how the resources will be allocated among the ones that do go. The guy so far has a decent track record on making those types of decitions.