Boss visited the night shift and found a car in the data centre Another week is coming to an end and that means it's time for On-Call, The Register's Friday column in which we share readers' stories of being asked to do the unthinkable to satisfy the incorrigible. This week, meet “Glenn” who told us that back in the 1970s he worked for “a large multinational that had several IBM mainframes …
> Motorcycles, bicycles, ski and other sports equipment - aplenty.
I knew a mate once who objected to being grounded to a deskjob so decided to park his motorbike on his desk. I bet it sounded good while he rode it up the fire escape.
The place I was working at the time we tried to work out whether we could get a mate's Mini up into our lab. Doors - check, space to turn it - check, goods lift- check, turning in to the corridor - check, getting it past the radiator by the kink in the corridor - Oh nuts!
Well it was a nice idea.
Back in the day at my residential halls in Southern NZ there was a tradition of pranks. One guy came back from a short trip away to find his car (memory says a mini but might not have been) in his room, on the second floor.
It had been disassembled then reassembled in the guy's room. He had to reverse the process to get it out.
You could do that to cars, and motorbikes in the day, they were user repairable. I can't even get the spark plugs out of my wife's car.
Impressive, but the gold medal still goes to the Cambridge University engineering students who managed in 1958 to hoist an Austin Seven car on the roof of the Senate House building one night. The full story is here, with a photograph of the car on the roof:
http://www.telegraph.co.uk/news/uknews/2210696/Cambridge-Universitys-1958-car-on-roof-prank-secrets-revealed.html
As someone born and brought up in Cambridge, I knew how this was done from only a few years later. I bet there must have been thousands who knew,
However, it was a brilliant plan, fairly well executed.
However, nothing beats the (much earlier) prank in which a bunch of gas board workers digging a hole in the road were told that students dressed as police were coming to harass them, and then the police were told a bunch of students dressed as gas board workers were digging up the road. Cue major fracas!
This post has been deleted by its author
Cars on the roof? Such things are routine here in the land of the free with too much money and time on their hands. Given the low academic demands on university sports teams who can supply the muscle, the availability of cheap wood and nearby forests in which to construct the ramp, it's no surprise to find the dean's car on the roof sometime towards the end of term.
Such things are routine here in the land of the free
There were a few classic VW Beetles around when I was younger.
Since it only took five or six reasonably able teens to lift one, they always seemed to end up hidden in hedges, casually sitting on decks, placed bumper to bark between any two convenient trees, or any other odd place that they'd fit.
Never put saw one on a roof though, a bit too heavy to lift that high. ☺
A VW Beetle decided to roost on the roof of my high school in 1979. Sat there for weeks. We also had a visit from a life-sized fiberglass Stegosaurus "borrowed' from a mini-golf place 25 miles away.
Irrelevant bonus trivia: I talked with one of the dinosaur guys later. They had slipped into the closed park at 2:00am, wrenched the Stego loose from its mounts, mahandled it over a fence and into a pickup truck. Some of them sat in back to stabilize the thing. So far so good! Of course with no traffic anywhere the very first traffic light catches them. So here's a truck filled with teenage guys and a dinosaur stopped at the light... and a local police car pulls up across the otherwise empty intersection. The cops look at the truck. The kids look at the cops. Anyone would know the dino belonged 100 yards up the road at Jolly Golf. The cops look at each other... and burst into laughter. The light goes green, the cops drive away. Awesome.
"In Durham legend has it that some engineers once suspended the VC's car from Kingsgate bridge"
That it was the VC's car sounds like a recent embellishment to me, but, a car, hanging from Kingsgate bridge? It would appear so.
https://community.dur.ac.uk/j.d.little/kingsgate/viewtopichtml_files/414.jpg
Not as ambitious as a car, but in school a teacher's yellow scooter was hoisted up with pulleys and parked 20 feet up on the edge of the sports hall roof.
When the teacher saw it she angrily stormed off and by the time she came marching back with the deputy head and a posse of other teachers the scooter was parked back down in the car park.
This was also done by the students at University College, London, about the same time. Engineering department students rigged a sheerlegs on the roof of the Engineering Building during the day, when it was not unusual for students to be up there, and one night they pushed the Austin Seven (about 1928 vintage, I think) up to the blank end wall of the building, hooked the hoist onto its front axle, and winched it up onto the roof. It was then manhandled across the roofs of that and other buildings until it was left perched crossways above the main entrance portico. College authorities had to employ the army and a huge crane to lift it down as they never found out who put it up there.
Many years later, at a College of Engineering in the Midlands where I was studying, one of our classmates woke up one morning to find his beloved Beetle perched on four (empty) beer barrels outside the Principal's Office.
...Also at a NZ uni student party, a bunch of us guys got a little leery and decided it would be great to pick up and relocate a little yellow Mini belonging to one of the girls at the party - between the two trees out in the front yard. A half dozen of us lurched outside, huffed and puffed for a bit - job done! Mini sat neatly in the middle of the lawn with front and rear bumpers nestled between the trees. Great fun!
Actually no ... none of us in our state of inebration had thought about the reaction of the owner which was to immediately burst into tears and hysterics, followed by the other girl friends flying into a rage calling us all the names under the sun until the car was carefully restored to its spot out front on the road.
Oops.
We tried similar with a Fiat 126, very easy for two strapping lads to lift the front up and walk away with it like a wheelbarrow (the handbrake was non-functional). Stopped when the front door was found to be too narrow due to the iron gates, didn't want to risk scratching either it or the gates.......
a 500 would have been about 4cm narrower ...
I devised a way to bring 500s with fucked engines back to life by transplanting a 126 engine and gearbox combo. Not as easy as it sounds. 126 had synchromesh, whilst 500 (which I learned to drive on) didn't. So I learned to drive with double declutching (before progressing to driving without using the clutch).
Does an Austin A30 count? The car could be completely stripped and rebuilt with two sizes of spanners and two sizes of flat-bladed screwdrivers. It was just about the ultimate in simplicity. As students, we disassembled one in the car park and took it - piece by piece - upstairs to a fourth-floor laboratory, where it was reassembled.
The following morning, the Prof admitted that it was the best April 1st prank that he'd ever seen when he found it between the benches in the lab waiting for the first lecture....!
Water, water, lots of water and live electricity.
FWIW no electrical shocks, but the water got warmer the nearer I went to the live plug (had no wellies and we need to down the servers and UPS'es).
Suffice to say I skedaddled out of there ASAP. Sparky need to look at the earth leakage setup.
Yeah one year after the Christmas party the on call (and therefore sober) staff member went to answer an alarm. On entering he noticed that the air temp was much higher than normal and that all the chillers were switched off. Whilst investigating this he heard noises coming from the other end of the room. There he found two far less sober members of staff down one of the aisles who were clothed as nature intended on packing blankets. If the floor had been covered in something different their actions could have been described as "Rhino on the Lino". They were so into each other that they didn't notice him standing there and then walking past.
They'd obviously thought that they'd found the perfect location to get "Jiggy With It" but it was obviously too cold. Their mistake was turning off the cooling which in turn alerted us that there was something wrong. The first time the pair paid any attention to the world was when the large yellow and red chiller master switch was switched back on. He didn't have to wait long for the temperature to drop a few degrees and for there to be a reaction. A figure covered in a packing blanket around the waist appeared and then disappeared the second they saw someone else. The on-call staff member yelled that they had two minutes to get dressed and get out before he'd be back with a camera (no mobiles allowed in there). When he came back with a camera they'd gone and the only evidence they'd been there was a scrap of packing blanket that had caught on a nail by the door. Everyone knew who it was because 'someone' had leaked the access logs for the security doors!
Yeah, one of the more fuckwitted people I've ever had the misfortune to work with. It's a surprise to me that he ever managed to remember the Ops Room door code, but it's not a surprise that he managed to unplug the wrong minicomputer. You'd have thought the flashing lights would have given it away, but no...
but it's not a surprise that he managed to unplug the wrong minicomputer.
It could be worse - he could attach a completely untested string of DASD units to the live production system..
Up to that point, we didn't realise that write-only memory was a thing. The writes would appear to succeed but no data was actually written. And we didn't find out for a day or so.
Said ops person rapidly became free to pursue other exciting opportunities.
Ah, but were they electric windows, or handle wound?
Used to know someone who, amongst other things, acted as a middleman for various manufacturers trying to get their products to other companies to sell to end-users.
At one point, he was going to one of the Scandanavian countries in order to try to sell some new double-glazing technologies to window installers there. Not wanting to let it go through the tender mercies of baggage-handlers, he wanted it in his carry-on luggage (it was a quarter-scale full working model).
His first mistake was assuming that the SAS checkin staff had a sense of humour. Apparently, giving the reason for having it in his carry-on luggage of "on plane flights I like a windows seat so I generally bring my own" and "I like to open the window and smell the fresh air" didn't go down too well and he spent several hours explaining his sense of humour to airport security.
This was before 9/11. He'd probably be on a plane to Gitmo if he tried that now..
Speaking of windows bashing...
The most unexpected thing I have found in a datacentre is men with buckets and sponges attempting to squeegee some allegedly breakproof film onto the windows in case some local chav decided to swing at them from the small gap between the window and a concrete retaining wall.
They set off the underfloor flood alerts and I find them and the duty security guard attempting to pat things dry in the subfloor with rolls of blue paper towels.
I've come across a whole backup IT system (completely outdated and hopeless underpowered with no way of shifting from the live system to this redundant system) kept in the server room 'just in case we ever need to replace the whole lot in a hurry'...
It's also worth noting this was for a computer parts distributor so if came to that we could just....you know use the stuff on the shelves if it was a big enough emergency...
(though it did have a number of 3Dfx cards from when they were a thing)
A friend and I were reminiscing just last weekend about the 'click' when our Orchid 3DFX cards kicked in. That was when you knew something glorious was going to happen!
Besides that, there's now a Glide wrapper so you can force the 3DFX calls to Direct3D for newer cards. I was playing Dungeon Keeper 1 last night. :D
Middle management.and higher (applies to management track folks only).
Any corporate bod with a non IT related Doctorate.
Tour groups.
Vendors.
Fast so-called food, and especially popcorn.
Mace/pepper spray (don't ask).
Last, but not least: Janitorial staff/cleaners.
Beyond the risk of being caught oily-handed, I'd have thought one would need to spread the weight of a Mini (kerb weight somewhere north of 600 kg) across more than four floor tiles, especially if one is thinking of removing a tile or three and then of getting underneath. Hands up who has seen a collapsed raised floor?
The old HPC data centre at Cambridge had racks positioned over the beams in the floors and also had the air flow calculated so exactly that they could track people in the room from their monitoring systems. It was a very impressive use of a space that wasn't really suitable for its purpose. They now have an excellent new facility for their beasts.
I've seen (and won the bid to fix!) a collapsed raised floor after the 1989 Loma Prieta earthquake. Belonged to a company in Silly Con Valley, next door to MaeWest.
That said, at least a few of the raised floors would easily support an F-250 full of line printers, card readers, tape drives, and the like. You can guess how I know this.
I'll second that with a similar story.
Large pre-built system delivered in loading bay (big enough for trucks) wheeled down concrete ground floor corridor to over-height freight lift...
Can't wheel the effin thing out of the lift landing as the doors to the office are standard height. the site survey guy didn't account for the fact the rack has to be LIFTED OFF THE FLOOR in order to move it around...
A friend once got called in to a computer that had gone down. As in, down through the floor to the room underneath.
Turned out that the disk packs were still readable. On the way down the power cables had pulled free, and the disk drives had noticed the loss of power and parked the heads while in mid-air.
This post has been deleted by its author
I haven't seen a collapsed floor... however. the story can now be told.
I came close to seeing a collapsed floor from below...
A cerrtain young engineer (might have been yours truly) was installing a new supercomputer at Nottingham University. Twelve racks full of Sun Opteron 1U servers.
Said engineer (OK it was me) was in the machine room to run cables between the switches at the base of each rack. So I took up the floor tiles behind the row of racks. The entire row.
The machien room manager (hi Chris) ran into the room, saw what I was doign and hauled be out by my ear. Thanks Chris. You stopped me from becoming a rather squished eingineer. Ever since I have had a healthy respect for tons of equipment balanced on machine room floors.
Ever since I have had a healthy respect for tons of equipment balanced on machine room floors.
One place I worked, we bought a number of (fully pre-populated) racks of Compaq servers. Sadly, the company that were (supposedly) a professional company, had managed to build them all with the (long-life and therefore very heavy) UPS units at the top of the racks.
We forced them to send their build staff to our site so that they could redo the build. They tried to charge us, we countered with the threat of a full HSE investigation.
Strangely enough, the rebuild happened at no charge to us.
A company I worked for had terrible trouble with overnight batch runs, almost always failing by morning. A new Ops manager took over, and discovered that the after-hours phone number given out to contact the operators actually rang at the payphone at the pub, fortuitously almost next door (this was in London). The operators spent the time until closing time in the pub, then attempted the overnight batch runs, with varying degrees of ethanol-fueled success.
The new ops manager soon put a stop to that. All was well with the batch runs for a while, but then other problems started arising, the mini-computers showing odd failures. The faults were traced to overheating, and the new Ops manager duly lifted the floor to place sensors, or some such reason. There in the under floor cooling ducts were crates - and I mean crates, not just a few bottles - of beer.
After a replacement set of operators arrived, all the batch problems, which had been a "feature" for several years, were resolved.
In addition to my main job, I'm also the health and safety rep, which equates to an excuse to just walk around looking at things now and then.
One of my finds was after someone had installed a new network link downstairs, which the reg standards converter tells me was a c-cup sized lump of masonry just dumped in the middle of the floor.
Needless to say official reports were written in triplicate, rolled up, stuffed with martian megaweed and the navel fluff of supermodels, and set on fire.
The lump of masonry itself was unceremoniously dumped out of the nearest window.
This post has been deleted by its author
Funny this comment should pop up today, because while I'm not having a server room problem, I discovered last night that the cleaners are storing waste paper in a fire equipment cupboard at my current workplace.
I have logged this with the very top of our "security and safety" department, alongside a lecture theatre with both fire exits blocked during exam sittings.
Not a happy bunny today.
Not a server room, but an amusing health and safety story:
I used to work for British Rail, and we had a room for wheel-tappers, shunters and other out-door workers to "rest and recuperate" as, before the days of globule warming, it was not uncommon to have to dry out a bit after work in heavy rain. (Or sleep on the table during night shifts - despite 5,000hp diesels parked with engines running 3 metres away).
The Health and Safety inspector came into the Area Manager's office and reported having found an electric kettle, plugged in, and switched on, on the gas stove, with the gas lit. The culprit admitted the offence, saying in mitigation "I was desperate for a cup of tea" - which was considered reasonably acceptable in the circumstances (after he made tea for all present).
I once was bivvying whilst out with the REME. They decided that they were going to run a Live fire exercise during the night to surprise us grunts. Apparently I slept through it until the the SM hauled me out of the bivouac by my feet.
There were grenades, smoke bombs and small arms fire just feet away.
After a very tyring exercise as an aussie, ARes infantry platoon sergeant, i copped 6months extra duty for sleeping through a tri-company attack on the same feature (better known as a battallion attack, which we were not allowed to be doing) . 500 soldiers (including my platoon), hueys, 40 ish m60s, artyand mortar sims, and the quack playing the bagpipes. Luckily, my extra duties only lasted 6 weeks, until I was commissioned.
Even desktops only need to be tested about once every 5~6 years (this idea that everything needs to be tested once a year is not true).
New machines/cables do not need to be tested if they are genuine.
So in reality PCs will often be replaced before they need to be checked. You still need to do a visual inspection and ensure that every member of staff knows to report and potential damage.
Golf Clubs, a Stack of fancy (Glass) bottled water and a rotting plate of food in one server room.
A LARGE Umbrella, this was on the top of the rack as the roof leaked.
An inflatable chair, kettle and ashtray.
A small child
9 year old HP servers running mission critical services...
The DC at my first job had a hand-printed sign on the door: "Anything left in here without my approval just became mine to use, abuse, and dispose of as I please" and signed by the lead night supervisor. Apparently some people, mostly not from the department, had been storing all kinds of things in the DC. This quickly stopped.
Not in the server room, but one of my colleagues used to work on his mini in the barn we rented for noisy engine tests. This was after hours and in his own time, so not an issue. Until one winter evening, whilst removing the radiator, his hand slipped and his arm got jammed between the radiator and the front of the car. Luckily, the hand-brake was off, and he managed to drag the car across the barn to the phone, and contact a another colleague who came and extracted him.
When I was an apprentice a mate rebuilt a Land Rover using the facilities of the material stores and replaced every bit of aluminium. His conversion to run it on Calor gas needed my assistance, as he was drawing off the gas so fast that the gas bottle was freezing. My solution- get a reel of 23/0076 wire, wind the cable round the gas cylinder and slap it across the battery. Eventually we worked out a control system (a switch) to stop the battery going flat.
Working at an ex-steelworks, re-purposed into something much more "friendly", a colleague built a pulse jet engine and attached it to a go-cart frame he also built. He had to buy in his own stainless steel, but most of the other bits, including the half-empty bottles of gas, were found lying around the site.
It did work, though it was a pig to start. We found that the best way to start it was to light it on gas and let it warm up, then upend the bottle so that it was running on liquid, while also blowing air through it.
Still not sure how we managed to get the specialist welding gasses past the accountant :-)
M.
When I worked the night shifts at a steel works, the other employees and I had the best serviced cars in the company. They were driven in for full services, wash, polish, hoover (courtesy of the office cleaner). They were even lifted up with fork lift trucks so that the underneath could be undersealed.
Pigeon nests, complete with pigeons, young and old.
Some bright spark had had the monster Airedale AC unit removed when it failed, it didn't need to be repaired as they'd decommed the huge DG mainframe it was specced for and the cooling solution in place was plenty good enough for the WIntel boxes.
Unfortunately they'd left a gaping hole in the DC wall which the local flying rats had taken advantage of so there were racks streaked with tokens of their appreciation.
Ah, asset tags. My employer is a subcontractor in an outsourcing contract, so some of the equipment in the customer's datacentre belongs to us. The lead contractor don't appear to understand this, so on a semi-regular basis they send a team of badly-shaved monkeys in to the datacentres to proclaim their ownership of all of our equipment by placing stickers over the inlet grilles to the cooling fans.
Many years ago on a night shift in TVCentre, I took my cycle inside for security reasons. It had a little speedo which showed max speed. Being not so busy, we got into a contest to see who could achieve the maximum speed in the central corridor of the area where we worked. All great fun until we spotted the black stripes on the carpet caused by over zealous braking and back wheel lock up*. The remedy? - swap the carpet tiles around to break up the long skid marks.
*not me, I know the most effective way to stop quickly.
A few years ago the company I worked for at the time was taken over by one of the large multi-nationals whose only expertise these days is making staff redundant, replacing them with more expensive contractors and paying obscene bonuses to the CEO.
My team got told one day that we were now responsible for supporting a few large server rooms in one of the UK offices. Basically, the company had laid off all the permanent staff, leaving a couple of contractors to do the work. The contractors were then let go to reduce cost. I met the last one to go on his last day and had a 20-minute handover and told that there was no documentation at all.
When I looked around, I found one of the server rooms had barely-functioning air conditioning, so somebody had decided to open all the external windows. There was a load of bird mess all over the floor and racks. I guess a warm server room makes a nice place for a pigeon to nest.
The design of a former employers datacentre was laid down prior to the widespread deployment of virtualisation, but completed after - as a result about 6 rows of 2 large 24 row data halls were occupied.
The second of these hadn't even been fitted with racks, so hows a CIO to use a cavernously large empty room? Why, as a space to learn to fly his RC helicopter and drone of course!
Following an issue with the air con, the head of H+S found an acetylene welding welding set at my previous job. A PM from the Home Office said she had to get some contractors to remove all the miniatures they were taking from their hotel and 'storing' in a comms room.
Some comms rooms I've had trouble getting into even when expected/correct paperwork/etc, one of my colleagues had trouble getting out as you needed a pass. He turned stuff off until they came to check and let him out.
A story about something that looked like it shouldn't have been in the mainframe room, but actually should've, sort of.
Years ago I did sales engineering (ie making excuses to the client about why our kit didn't work) for a protocol converter company that made System/370 compatible printer converter boxes. One day I went to a large electricity board head office, where one of our printer boxes was playing up. The mainframe was sited in a large room, with a Heath-Robinson tent over it made out of wood and plastic sheeting.
I asked why it was there, and the bloke in charge said something like,
"Well, we couldn't afford a proper fire extinguisher system, so if the building catches fire and the sprinklers go off, the tent'll keep the mainframe dry. But if the computer catches fire, it'll burn through the plastic, and the sprinklers will put it out!"
That's proper engineering.
We found the same Corvette hidden away three times.
The first time it turned up in the warehouse during quarterly inventory, hidden under empty Compaq boxes and a tarp. After asking around it was determined that the car belonged to one of the operations guys and had been there for at least a month.
He was promptly told that "No, it's not okay to store your personal things in another department's space, and besides, we don't like flammables and solvents stored with delicate computer equipment. Get it out of here."
Months later it turned up again in maintenance storage off the parking garage. Normally it wouldn't have been a problem, if he had bothered to ask, but he hadn't, so out it went.
The Corvette spent a week in an outside lot before disappearing again. This time, he was crafty. He went to his boss, complained about having to store his car elsewhere, and wanted permission to move some 'personal belongings' from his garage into furniture storage.
His boss wasn't expecting him to come in on Saturday to roll the car into the main lobby, through several sets of double doors, across half of the sales office and into said furniture room. But that's exactly what he did.
When the car was discovered a third time by someone checking emergency lights it was the end for him.
... required a regular oil refill. That was messy
I'll see your oil fill, and raise you ... a (IIRC) Ricoh duplicator that had a fault which meant that it couldn't automatically unload it's masters.
For those who have never seen such a device, it looks just like a photocopier - in theory, walk up, put your original on top, press the copy button, out comes a copy. But, this uses a sort of thin paper with plastic coating to make a master which then does the copy with ink - real ink, in paste form. Once the master is made, you can then rattle off hundreds or thousands of copies very cheaply compared to a photocopier. The one we had would do 130 copies/minute which was way way faster than any of the photocopiers we had as well - but fun when the output tray wasn't quite set up right :-)
With this machine, it was supposed to be able to remove the inky master from the drum and deposit it in a waste hopper when you made a new one. But as mentioned, this machine had a fault, so if you didn't manually remove the master first (a clean operation, as part of it never got inked) it would simply shred it and wrap bits of horrible inky paper/plastic round the mechanism.
No matter how many times I explained this to certain people, they would never accept it - and somehow it was my fault, and my responsibility to clean it out. Oh no, those dainty (well some of them were at least) ladies couldn't possibly get their hands dirty doing such a messy job.
That was just the bodywork.
The Seniors at Cubberly High School (Palo Alto) welded a complete VW bug/beetle onto the top of the School's Totem in the late 1960s. The car was running when the staff got to work in the morning. You can still see the tabs the car was mounted to welded to the top of the supporting i-beams in the picture, taken over 40 years later. I can't find a pic of the car on top online, but I know they exist ... I think I still have access to the negs.
Not to be outdone, some ten years later we disassembled the Principle's own VW, and lovingly reassembled it in his office. He walked into work, to find it idling where his desk should be ...
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2025
