
So the penultimate paragraph sums it nicely 'you don't need logic!'. And there's the bear trap. Your AI sees a pattern - you don't know how it found it, it can't explain. The pattern might be real, it might not be - its almost certainly pre-determined by the 'data' you fed the AI - and data in the real world is never 'clean' and 'un-biased' and 'representative'. Even if the training data was immaculate, you may or not be able to predict anything useful. So my 'beat the odds' AI analyses a billion coin tosses - it can tell me the probability of heads - it can tell me nothing about the next coin toss.