Re: The AI will just...
Its not beyond hokey when you look at those models or real AI being given a reward for completion of a task but not being given strict enough instructions not to cheat. Yes, its highly unlikely if we take AI safety serious enough but also not impossible.
And you can never say true General Intelligence AI is impossible. Its highly improbable at the moment but we don't know what we'll invent in the future that may make it possible.