Keep your cables tidy. You never know when someone might need some wine We've all heard the one about the cleaner in the hospital pulling the wrong plug. But managing to push back the schedule by months through sheer brute force? Welcome to Who, Me? Our story comes from a reader Regomized as "Tom" and concerns the time his employer, a major parcel delivery company, was planning to open a new …
No yobbo's or lookie-lou's allowed.
Erm, yes. That is, until I came across that client: the complete staff was allowed to enter its server room. I did not like it and asked them to restrict access to those who actually need to access it. And their response: "We did." - ??? - "It's the coolest room and we store the beverages in there so everyone needs access."
I hope whoever agreed to allowing catering to use the server room for cooling received their pink slip before the day was over. Dear lord that's a horrendours decision.
Who could ever think letting a load of unknown, low paid, catering staff to run loose in your brand new server room was ever a good idea. They maybe saved $100 by not having to hire a refrigerated trailer to store there stuff, and look what it cost them!!!
My Flabber is well and truly gasted...
At a previous company I remember walking into one of the comms rooms to find I couldn’t get to the network switch to trace a fault because facilities had filled the room up with pallets (yes plural) of bottled water.
The water was for a promotional thing to the staff they got to told to remove it and the question came back why was it a problem ?
I left that for management to deal with
Who says anyone "allowed" the caterers to use the server room as a fridge? It is entirely likely that they, or someone low-life they talked to, made the decision themselves without bothering to consult Those Who Would Actually Know Better (tm). The real question in my mind is - why wasn't the server room properly locked down with some serious access control?
Or the building designers put an aircon component that requires maintenance in a roof space only accessible from the server room...
And the server room rarely requires access by IT staff it may be some considerable time before you discover that the Facilities staff have decided this is a good place to put Christmas decorations with disintegrating metallic tinsel...
And if you suggest that the Facilities staff need to relocate the decorations in the manner of the good fairy and the Christmas tree then you may find yourself in trouble with HR
Don't ask me how I know...
"I hope whoever agreed to allowing catering to use the server room for cooling received their pink slip before the day was over."
It was described as a "server closet" and "at the turn of the century", so I'm guessing we aren't talking about "server central" and BOFH + PFY roaming around. More likely a rack or two and barely room to swing a cat.
"Dear lord that's a horrendours[sic] decision."
Despite the above, I still agree with this bit :-)
It all depends who came up with the idea. I've worked plenty of places where an otherwise sensible IT decision (eg, 'don't use the server room as a fridge') would be overruled by someone higher up. After all, if they get paid more money that me, then they must know better right?
Of course, when said idiotic decision turned out to be a bad idea, it wasn't the execs that took the blame....
("why didn't you tell us it was a bad idea", 'I did, here's the email', "I didn't read that, you should have told me in person", 'I did!', "well, I don't remember that, so it's all your fault").
A multiple-month delay thanks to a bit of careless footwork on the part of a caterer
Not a fault of caterer for being a clumsy caterer. The blame is on anyone responsible for the server room security.
Also when you run a server room yourself, you need to have some redundancy. I know servers are expensive, but that's just how it is. Even one spare server would get them out of trouble.
VOIP Telco in early 2000's (startup so many ethical corners cut - oh the memories...)
wanted a local gateway in a particularly high volume destination of our phone cards (remember them?)
anyway... official telco licenses were impossible, we had support of a local general, who was willing to house equipment at his house and arrange the local E1 drops - for a rental fee and revenue share deal.
so we shipped out half a dozen preconfigured AS5300's they got escorted through customs to the generals house, and appeared on-net rapidly. Traffic flowed and the backhanders seemed to have paid off.
until the rainy season.
boxes went down constantly. came back up, but were particularly flaky.
we sent out an engineer to "get it back to the reliability we had at the start" on landing at this tropical hellhole and getting transported to the generals pad was pointed to the stack of boxes in a corner of the veranda. he took a picture (with a real camera) of the cables coming down the building acting as a perfect rain conduit directly to the boxes.
asked if they could get moved in to the house until the rain stopped was met with grumbling... made comments about revenue share needed the boxes up to work...and it happened. they called in a local labourer to literally knock a hole in the wall so the cables could get fed through (hole stuffed with newspaper afterwards to "seal") . once in the lounge the kit worked without issue
eye opening all round.
Sometimes you have to be pragmatic either fly the parts out with an engineer with a completed low value customs clearance form and a pocket full of cash, or have a conversation with the customs agent locally about the additional inspection charges required to release the kit.
Its wrong, irritating and probably illegal locally and if you can plan for delays without interrupting progress then do so, but the cash amounts are normally so low they don't even appear as a blip on the project budget.
the whole import & export of hardware can be fraught with entirely legal difficulties, never mind illegal ones. I've heard of servers & laptops being confiscated -legally - by governments. And we're not talking third world governments here: I'm sure the US was one of the worst culprits.
I remember being told about the connection between 2 mainframes - the cable was a maximum of 5 metres long. The usual configuration was ... 1 Metre from socket, down to the floor ... 3 m under the floor and 1 m up to the other machine. The customer's problem was the machines were 4 m apart, so it went across the room at waist height, over other kit and a chairback etc.
This worked fine till someone tripped and backed into the cable and pulled it out of the machines.
There was a bank of DASD - 7ft high . 3 ft square in slightly the wrong place (2 inches). They put their backs into it - and pushed with the legs... and moved the disks. As they stood up - someones's belt got caught on the Emergency Power Off button - which was pulled - and dropped the power. It took a day or so to recover!
Was at a client site in NW London in the 90s and had to go behind their server racks (reason lost in mists of time) into a total rat's nest of wiring. Needless to say, I tripped and yanked the power cable out of a UPS outlet that was supplying 'our' server. Sheepishly reinserted, but as I knew the server in question had been specified with dual-redundant PSUs I was quite confident of having got away with it. Until, upon emerging, the office is all a-fluster. They had decided to save electricity and unplugged the UPS for the second PSU as it was apparently 'redundant'.
Managing a community radio station in Ottawa, Canada. Needless to say by 1999 the radio control room was a no-liquids zone. Late one night I get a frantic phone call telling me that a snottty kid calling himself DJ T*****B*** had dumped a can of Coca-Cola into the mixing board.
I phone the twerp the next day to chew him out, and his very first words to me weren't "Gosh, I'm sorry." They were "It wasn't me, it was my girlfriend, here's her phone number. "
Followed quickly, when I explained that he was now an ex-volunteer radio host, by "You can't do that! I'm incredibly popular in Europe, and your name will me mud!"
* It seems that "DJ T****B***" is still floating around these days these days, so I'll anonymize him.
As I think I've recounted here before, I had exactly that happen at an ILR station in Cardiff back in the late 1990s. DJ in question was doing his last live show before moving on to pastures new and a well-wisher had sent him a bottle of cider.
Of course, it wasn't he who called it in at any point in his mid-evening (7 - 10pm?) show, nope, it was the poor sap who followed him who had to page me and listen to me moaning.
One thing which really annoyed me was that both jocks flat out refused to move to the spare studio. Granted this was in the days when they had boxes of CDs to shift, but the spare was literally five steps away and ok, yes, the "offer, accept" handover system was a bit awkward to do on your own, but not impossible.
Instead, both carried on with half a working desk, making do with the guest microphone, one CD player and the playout system if I remember correctly. Maybe the phone TBU too.
On the positive side, the desk surface was just some sealed switches, sealed pots and Penny & Giles conductive plastic faders; all the electronics were in a pod in a rack. The desk carried on without a glitch (apart from the faders covered in sticky cider), the drain holes in the bottom ensured that jock went away looking as if he'd had "an accident" and I was able to revive the faders by running them under a tap and replacing a couple of wipers that may have been close to needing replacement anyway.
But I could have done it at 8pm when the accident happened, rather than at 11pm.
M.
May have been, but it were nothing to do with me. My name refers to Myghal Josep an Gof / the Smith, who was one of the leaders of the Cornish rebellion in 1497... and we all know how well that went. The DJ in question's name escapes me right now (Chris something?), he was only with us for a couple of years and headed off to Leeds (Aire) I think, or maybe not. Our greatest claim to fame was probably Bobby McVay, briefly famous for the Eurovision song "I'm never giving up" and latterly as a "spare" fourth member of Bucks Fizz. In those days we had a thriving newsroom with close ties to the Cardiff School of Journalism, and several alumni are still active in the field.
M.
Back in the very early eighties, a new multiprocessor system was being demonstrated at a trade fair in Germany. It was literally the prototype - the only one that was working in the world.
The boss managed to spill his beer into it.
Every card was removed from the rack, and carefully rinsed with water and dried. The backplane was also carefully washed and dried. It was all put back together, with fingers crossed and a prayer or two offered up to the gods of technology. Power was applied....
It sprang into life, like it had never been away.
The old custom IBM Model-M (and even older Westinghouse) keyboards at BA LHR checkin had God-knows what poured into them (various bodily fluids included I sustpect..). Standard recovery was to put them, intact, through the high-temp industrial dishwashers from which they would emerge spotless and working perfectly. The later rubber-membrane stuff sadly didn't respond so well...
The time an unopened can of Coke spontaneously exploded, and the resulting splatter took out our only copy of the installation floppies for some mission-critical software.
Several fails, obviously: only copy; food on the same shelf; all of it by a window (don't know, but I'm guessing that the can was in the sun).
I run my own business so I suppose I am sort of an MD.
As for keys to the server room, (actually a cellar) why would I want to have a key?
I pay someone else to look after that sort of thing. It's not like I haven't got many other things to do.
Love the way you refer to the company that hires your services server room, as MY server room.
"Love the way you refer to the company that hires your services server room, as MY server room."
You appear to be unfamiliar with BOFHs. You are advised not to enter any stariwells or lift shafts in the company of any of your IT staff and to keep clear of any windows when they're about. Parking your car away from anywhere where a heavy object may fall fromm a building is also a good idea.
The tensile strength of a lot of cables at once can end up failing to pull out or break just the cable. Even one cable with a secure enough mechanism (E.G. the ones with extra pins just to keep the plug connected) can transfer a lot of the stress to the thing it's connected to. In addition, the strength of someone pulling with their leg is a lot higher than with their arm, meaning the stress on the backplane would have been one quick pull, whereas pulling the cables with your arm would have taken longer and given some of the cables a chance to break off and transfer the strength to others.
I believe a certain software vendor in the home counties had an impressive wine collection gathered by one of the two co-owners in one of the server/patch rooms for exactly the same reason. It was cool, it usually was dark, not many people had access (and those admins who *did* have access knew not to touch said alcoholic goods).
And apparently that arrangement persisted way past the day the owners signed their baby over to some investment bankers for a *lot* of wonga... until someone pointed out that since the baby wasn't theirs anymore, it wasn't appropriate for them to use said room in the building to store personal property. I don't know what happened next, but I suspect said personal property must've been relocated to a more suitable venue...
Back in late 90's at a national ISP, one of the sysadmins didn't see any reason to avoid alcohol just because he was on call. His favourite drinking place was just around the corner from the office anyway, so it's not as if he'd need to be able to drive there in case something happened!
Something did happen. He got back to the office and up to the server room... and stumbled, bringing an entire rack down.
I think he sobered up fairly quickly thanks to the adrenaline. I also don't think the customers ever knew the cause of that outage.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
