Here's a refutation: that's an approximation of the definition of symbolic and subsymbolic systems.There are plenty of ML schemes that are fully capable of explaining themselves, most famously expert systems. On the flipside, one conclusion from the stated definition is that virtually every person is at least intermittently not "intelligent".
<rant>And this is the weakness of at least the "public comment" side of such "debates" - people with (next to) no grasp of the subject reinventing the wheel with their late-pub-night grade hypotheses instead of spending the 5 minutes it would take them to actually improve their relevant personal knowledge. It is evident even in this very thread (no, Elledan, that's not how working on improving a CNN model works, unless done by a complete amateur that spent the last 5 years under a rock). In the end, the subject of the discussion devolved into "hurr durr, dem computers r dum cuz i says so".</rant>
Sorry for the abrasiveness there, but there's only so many times one hears something in the key of "heheh, it's just a a bunch ifs" before they inevitably blow their top - essentially the equivalent of "they just sit in front of their computers all they and get paid for it!" for ML specialists.
Biting the hand that feeds IT
