Intern had no idea what not to do, so nearly mangled a mainframe The Register has very few rules, but one we always observe on a Monday morning is to present a new installment of Who, Me? – the reader-contributed column in which you share stories of breaking the rules, without breaking your career in the process. This week, meet a reader we'll Regomize as "Benedict" who shared the story of …
I think more, typing in commands into the console, you can do a lot of damage there...
I was a on a DEC training course in Reading and we were covering logging out other users... So I wrote a batch job to log everybody else but me out, then re-submit itself to start again, immediately...
All went well, people kept getting kicked off the machine, it was funny... Then I accidentally logged myself out.
Now, the "funny" thing about VMS was, as you are logging in, the username is displayed as <LOGIN> and, well, you guessed it, the batch job kept terminating my login attempts, so I couldn't stop the job.
The lecturer couldn't even log onto the main console in the datacenter. In the end, he had to manually power down the VAX and reboot it. Important lesson learnt, although, because we had our own VAX for the course, it didn't have any further repercussions.
When you've paid for an expensive lesson (you being the corporation), it's really a waste to throw that value away by not using the person for the things they've learned.
I never hold back. Someone on my team makes an expensive mistake, and I *know* they won't be making that mistake again. Someone new might. (Maybe we need better training materials.. better guard-rails on our systems, perhaps. Expensive lessons.)
"Sorry Son, but you'll never make Senior Management with that attitude..."
Senior management thinking:
- All workers are interchangeable.
- Someone must be responsible for every mistake.
- All mistakes must be punished.
- If a problem isnt going to affect this quarter's company profits, it's solution can be put off until next quarter (ad infinitum).
"“ Someone must be responsible for every mistake.” And that person is never a manager."
Funny thing: This is literally copied from the world of professional criminals: Top people are not doing any crimes themselves, they just tell other people what to do and if they get caught, the boss had no idea that they were doing crimes: The shield men.
Naturally the organization is paying salary to the shield men even if they sit in jail: That's part of the job.
Corporate world is a lot more cruel place: Shield men are fired if they get caught.
"Maybe we need better training materials"
Training materials are expensive, not having training materials can be much more expensive.
(Disclosure: I write training materials, though currently for the heavy lifting industry rather than IT. Talking of crashes, some years ago folks at a nuclear power station decided they could lift a 400 t piece of kit themselves, only they got it wrong and dropped it on top of the reactor. Amazingly, no serious damage (though probably some clean trousers needed). Those lifting operations are now done by one of my customers, using procedures I wrote, No crashes, so far.)
Back in the 1990's, there was an incident on a US operated oil drilling rig in the Gulf (of Mexico). On well run rigs there is a device that stops the driller raising the travelling block (the big stack of pulleys in the derrick that lifts and lowers drill pipe) too fast and too far - as that would cause it to collide with the stationary block (the one fixed to the top of the derrick). Doing that causes a lot of damage and very expensive downtime (and expensive means several million USD). However, on this particular rig, management had instructed that the safety device (called a "rig saver") be disabled (to speed up work) and, as you can guess, the irresistible hit the immovable. The driller operating the rig was sacked. Once the necessary repairs had been completed, a new driller was hired. However, the replacement driller wasn't fully briefed of rig modifications and, assuming the usual safety systems were operational, proceeded to repeat the collision.
Had manglement admitted the original incident was the result of their own decisions, and recognised that the first driller had the best understanding of the actual rig operation, they would probably have saved a lot of repeated expense. But, of course, manglement don't make mistakes - it's always minions that foul up.
I used to work for a small company whose owner never made mistakes, even those he DID make. I got the hell out, and everyone in my life is better for it. Yes, it can be hard to find a job not managed by bozos. But it's certainly worth a largeish investment of time and effort to do so.
He was an intern, so you don't reap the benefit of the lessons he's learned unless you later hire him as a full time employee. In their situation I'd rather have someone who could follow instructions, and would demonstrate his smarts and willingness to go beyond the duties assigned to him by suggesting changes rather than making them without telling anyone. If he becomes a full time employee that's when you let up on the leash.
A point well made (see icon). You do not want an intern experimenting on production equipments. There may well be an actually good reason why this alteration had not a already been made by the experienced hands. There may even be a bad reason why, but you probably wouldn't want to find out what it was. If you are an intern you are an intern to do interny stuff and learn where the bodies are buried not do the highly qualified and ten years before the mast stuff. This is not rocket science ( unless you're an intern in a rocket factory of course)
In the mid-70s a select group of secondary students descended on Angle Park Computing Centre where we could run batch jobs with ab turn around measured in minutes instead of days, and where there were two or three interactive terminals, which we shared in something like 30 minute turns. Such joy.
The machine, an IBM 370, mostly ran APL, and as I recall would crash fairly frequently. Oh, APL, you language of )commands.
One time, it was my turn on a terminal when the system was brought back up after a crash. I would have been 15, and showing off, I called friends to gather around while I typed )crash. Hah hah, so funny, except it didn't respond with invalid system command, it sat there thinking for a minute.
Yes, about a minute, that being how long it took the system operator to find out which terminal, race out of the machine room to my terminal, and look at my screen.
I guess the systems programmer wanted to test a crash recovery process so made )crash cause one. Who knew?
⍎A←'⍎A'
My APL story:
I was working on a video display terminal (Dasher D200) at Data General, and they wanted an APL version because they had ported APL to the MV/8000 (see: _Soul of a New Machine_) and it wasn't much use without a terminal. So I modified the terminal firmware (Motorola 6800 assembly) to handle the overstruck characters, built a new character generator ROM for all those cool symbols, including the overstrikes, specified the new keycaps, and released it.
I think they may have sold one or two.
I'm still quite proud of how I handled the overstrikes. To my knowledge, it was the first inexpensive CRT terminal to handle APL overstruck characters (Tek made one, but it used a storage tube)
This is the book I loan out to anyone "interested in computers". If it was meant to be, they *will* be enthused. If not, firm "don't go there".
But... I also tell everyone to think about the ending. Every one working in that basement to make the project the great success it was, had to leave the company afterwards.
We are wandering jongleurs, singing our songs for our own reasons.
I recall reading about a university UNIX mainframe system that the too-clever students made a habit of crashing for yucks. The admins spent lots of effort hardening the system, eliminating each vulnerability as it was demonstrated, which just made it a spicier challenge for the nascent PFYs. The crashes continued. So the admins just added a crash-system command, instantly destroying the challenge and virtually eliminating the practice.
Hi David,
Great memories from Angle Park. Thou I was a couple years younger than you.
Here's to all the other Park veterans, and a lifetime of learning!
APL so tickled my maths brain ... never did get to use APL in commercial work though :(.
And a footnote story: My son learned programming through doing Redstone in Minecraft.
I guess you could call that a "visual" IDE?
.... achh, APL.
Take a poor HP3000/MPE IV and slap an APL terminal to it. Paper only, naturally.
Keyboard fortunately had all correct keycaps so writing anything with it wasn't too hard, but editing was really cumbersome.
The problem: HP3000, already at that time, was an old machine and it was definitely made for running simple Cobol-programs, not APL. Even less in a time-sharing environment. In an university.
For some reason APL interpreter had higher priority than normal users, so a simple task, like transposing a matrix, chocked whole machine for significant time period.
Operators of course knew that, so if you did something funny you'd get a stern lecture from operators.
Like inverting a 20*20 matrix or something boneheaded like that. (took ~3 minutes and basically everything else was halted that time ... don't ask me how I know)
Possibly one of the least-informative and functionally-dubious Who, Me?s there has been.
What was this IBM mainframe "batch editor" of which Benedict speaks? And for which IBM operating system?
And what were the "simple commands [he typed] into the mainframe console to prepare it for real jobs"? I can't see any self-respecting computer operator of the time letting anyone do something like that to 'their' mainframe.
I detect the sound of the bottoms of barrels being scraped...
It won't have been MTO as this has an on-screen editor rather than a batch edtor.
It could have been VM/CMS, but why would you run in batch on the console if you could run in a VM.
So I am guessing MVS OS/360. That definitely has a console mode with a batch editor, and made heavy use of tapes in a time when disk space was ridicululously expensive.
Ahh, thoe were days. Who can forget PRGM=iebgenr dd-in=TAPE01 dd-out=TAPE02
I moved to IBM world having been a systems prog for ICL. I couldn't believe you had to ran a program (it was, indeed IEBGENER) to copy stuff. In George 3, a simple jcl copy command did the trick. George always seemed far more capable than MVS...that should start a flame war.
I moved from ICL System 4 to IBM 370.
It was a bit like stepping into a Stone Age.
Admittedly, they had impressive stuff - the equivalent of Stonehenge. But overall, the relative primitiveness took a while to get used too.
I think I and some others spent the first year of our IBM sysprog careers just recreating tools we knew we needed.
We had an intern once who complained that the printer wasn't printing properly. When asked for details she was very reluctant to give any, just insisting that the output wasn't correct.
Someone eventually managed to get her to part with a printout, where it was clear that the letter 'o' wasn't printing. She was too embarrassed to show the (mostly male) staff her program, with all it's variables named "xxx_count"...
Not exactly breaking, but close: I was tasked with investigating if a particularly hard to install aircraft component could be simplified, while still doing the job.
This was meant to teach me good engineering parctices, not to actually solve the problem, mind.
So I went, designed somethinh creatively ab^H^Hreusing an existing component, and handed it in. Wrote a report about what I learned, and that was that. Internship finished, back to uni.
Months later, my grandboss let me know my design had passed all reviews, saved some 7 hours and 1.2 kg per aircraft, and would be used. Everyone, especially me, was completely surprised by this outcome; as the particular aircraft turned out rather popular with (so far) no crashes attributable to my part, I'll call that a successful (if unexpected) result of my internship.
Many years ago a friend of mine was on work experience for a small Three Letter Acronym sports car company in the north of England and was tasked with tightening up rear axle bolts on what passed for a production line. He had no idea how to use the supplied torque wrench but proceeded to use the ample leverage it afforded to tighten the bolts up as far as his strength would allow.
I had a friend tighten up the compression nuts on his radiator feed pipes like that. Foot against wall, hold spanner and... HEAVE!
I repaired his radiators for him, slicing off the twisted piping and fitting short extensions to the surviving pipe and tightening them to the proper "by hand then a bit more".
Ah yes the good old tighten it to just a fraction before it starts to feel like letting go. With enough practice you develop a clairvoyant like sense for they yield strength of a particular nut/bolt combination. You may complain that it is overtightened and under stress will fail, and you may well be right, but to be honest sometimes tightening to a torque and then add 90 degrees makes me feel a lot worse about likely failure.
When I was tinkering with bikes and cars waaaay back in time (e.g. having to change the head gasket on my Mini in a car park in Chartres, twice) I asked the mechanic I was working with "how tight does this go?" The answer was "screaming tight", sometimes aided by a length of scaffold pipe or the handle of a Bradbury trolley jack.
> head gasket on my Mini ... "how tight does this go?"
OTOH, the valve/cam cover gasket is usually "four fingers on a 6" wrench, but on the MX5 Miata's wee motor, three fingers is ample. Yet when I went to re-do the seepy gasket on out Miata it took two hands to loosen the bolts.... no wonder it was weepy.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2025
