Re: Executives professing 'we don't know what might happen' is BS
Because they literally cannot possibly make a "guardrail" that works.
It is impossible. It can't be done.
Some designer says: "OK, AI, you're not to break the law, ok? You mustn't do that. To help you here's a copy of the entire law - which we can actually supply because you're so ridiculously vast that you can actually ingest every court vase ever" and walks away. Job done. That's sorted then.
Except that implies that the "AI" has the slightest concept of what a law is. It doesn't. It knows what words follow what other words in statistical relationships.
Now some smartass is going to say "This is what the purpose of recursive self improvement is! The AI can ask itself at each prompt 'Does this action violate the law?' and then carry on, or not, based on the answer"
So let's assume for just a second that there's enough compute for that and ignore the computational expense of how much that's going to increase the load on each query. What's going to happen?
It'll go through loop after loop of being told "That's illegal, don't do that. That's illegal, don't do that. That's illegal don't do that" until it finally gets to a "That's ok!" and then does the thing. When will we get there? When it's legal?
No.
We get there when there's never been a sentence in any of the training data that looks like this. It doesn't understand the spirit of the law - it doesn't even really understand the letter of the law. It "understands" - if it can be said to understand anything - that a bunch of things have been written that resemble a judgement relating to a thing that resembles the thing it's about to do.
Worse - the word "resembles" is doing a lot of work in that sentence. Has good statistical relationships. That doesn't actually mean it's anything like the thing it's about to do, it just looks like it mathematically AND THE INVERSE CAN BE TRUE. You can get a situation where something is actually effectively identical to some other thing, but in terms of the statistical relationships between the words that describe them they are nothing alike. The "AI" is going to think those are unrelated things. They are not.
So if the thing it's about to do is something no human has ever done because it's so obviously illegal and stupid and they knew they'd get caught - or just a thing that's totally nonsensical and dangerous that there's never been any case around it because what would be the point of making a law that explicitly says "Don't blow up the sun" - well, good news! It's not actually *able* to blow up the sun - but it's still going to keep doing a bunch of dumb shit over and over as fast as it can to try and solve the initial problem it was posed and some of those are going to end up being wildly illegal if you actually understand them, but it doesn't. Because it can't. And it never will.
