Re: VMS?
Well done Steve.
You knew about the autoanswerback fun. Good start.
You used the operator broadcast mechanism (which needs OPER privilege, which by the sound of things somebody shouldn't have given you?) to send the VT100 (or any other terminal with programmable answerback) the escape sequence to program the answerback, and put something destructive in the answerback.
You then used the same mechanism to send all the lines the command string which causes the terminal to send its autoanswerback (was it Control-E, I forget?).
If a hardware-vulnerable terminal is not logged in, nothing much happens.
If a hardware-vulnerable terminal is in an application, the application sees some unexpected input.
If a hardware-vulnerable terminal is at the VMS command prompt logged in to a non-privileged account, the "shell" sees your command string and says "can't do that, you don't have the necessary privilege" (as you already described).
If a hardware-vulnerable terminal is at the VMS command prompt logged in to a privileged account, you may see something interesting happen, again as per your description.
Lots of luck in that, apparently including having an incompetent manager, and definitely no guarantees of success.
The same scenario potentially also applies to a UNIX/Linux session on a serial terminal (or emulator). It was a known area of concern at least as far back as 1999 e.g. ftp://ftp.cs.utk.edu/pub/shuford/terminal/answerback_news.txt says "If the terminal [emulation] allows the host to program the Answerback string, it can become a security hole."
This is really a vulnerability in the terminal not the OS, and iirc from DEC's VT200 series on (other terminals and emulators are available), autoanswerback was disabled by default.
Now compare with the popular desktop OS and its vulnerabilities: frequently no luck needed, excellent chance of success (if you pick the right security hole).