The cleaner did it.
It’s always the cleaner.
Welcome to On Call, The Register's reminder of happier times, when the only panic was the one coming out of the easily ignored telephone. Today's tale, furnished by a reader called "Bob", which is most definitely not his name, takes us back, Pulp-style, to when Windows Server 2000 was being rolled out to replace Windows NT 4 …
Except when it's the Hare...
Possibly apocryphal* but way back in the mid 80s** a system kept failing in the early evening once or twice most weeks. It was clearly a power issue but everything tested out OK. Eventually somebody twigged that the outages coincided with meetings at the next door dog track.
As told it was starting the hare running that caused a power dip, we suspected it was more likely when the floodlights were turned on.
*I was told it by a software guy from the supplier involved who'd got it from one of their field engineers.
**When UPSs were less common,
Worked once for a UK media business that was based in Central London. We had during the dotcom boom a plan to update and revamp our online offering quite substantially. To do this we hired some talented people and had our own kit separate from the main server rooms (by one floor) for work before it was installed elsewhere both in the building and elsewhere. A small room with very good aircon was chosen for storing the now powered up equipment because it was one of the few places where the floor was reinforced enough to take the weight.
We had a system for out of hours call outs, someone had written a small prog that alerted reception something had failed and they then called the out of hours mobile number. There was I think one call out in the middle of the first week when a cleaner switched off something by accident. These things happen but are annoying and so a High Voltage Danger of Death sign was placed on the door. The cleaning staff were also told not to go in and this was fine with them as they found it too cold in there. So we get to that weekend, I'm on call the phone rang and they said they're showing multiple failures. I got in (I was nearby) and found someone left the door open and things overheated and thermally cut out. This was during the summer so the aircon was working overtime and that had cut out too. I reset everything and went back shopping.
Next weekend I was on call again (lucky me) and the same thing happened, had a call from reception "multiple failures". I was a bit further away but I was able to get in quite quickly. This time the door's closed but the aircon is off and it's a far cooler day. Again I reset everything and went home and heard nothing all the rest of the weekend. This pattern continued for two more weeks when whoever is on call is phoned with multiple failures. This seems malicious now so I suggested locking the door, with the key being held on reception in the key safe for access if needed. Doesn't seem to stop it though and the key has never left reception so it's a bit of a puzzle. We viewed the security camera footage and no one was seen near the door so all very mysterious.
The next weekend I'm the unlucky sod who will get the call and I realised from reviewing the call log on the phone that they're all coming in at roughly the same time ±5mins. It happens at around 6pm on the Saturday and I decided I'm going to catch whoever is doing it. I got in at 5pm got the key and unlocked the door the plan being to wait for the miscreant to come in and I'll surprise them. I take a book and a chair and settle down all murder tv show style to wait for the handle to turn. At 17:30 there was no handle turning but mysteriously the aircon died and the room started to warm up. I started digging into the aircon panel options and discovered that whilst there isn't an auto on time set, there is an auto off. No one (and certainly not the person who was given the responsibility) had bothered to check if the auto off was set because the auto on wasn't.
The person concerned who was supposed to check was made the "voluntary" on call person for the rest of the time we had the kit on our floor. Then the week after it had gone to the main server room etc, this person was still in possession of the phone when they should have handed it back. As it was a weekend the boss called reception and asked them to call him and say there were "multiple failures" showing again. He made it halfway in before realising that he'd been had.
(I've told this before...)
We had a 'client', a wine merchant, who regularly reported intermittent problems with his leased line link early in the morning but by the time an engineer turned up the fault had gone.
One day he phoned to say he had sussed it... it seemed to be occurring when the café next door was opening, so he guessed they were causing a mains spike
Next day we turned up early with the normal test gear plus an expensive mains monitor borrowed from R&D. We sat back, sipping a very agreeable sherry (at 9am?!? well the café hadn't opened yet) waiting for all hell to break out.
The café opened as usual, but nothing!! A few line errors but nothing to worry about...
Then the guy, as he did every morning, got up to fetch a ledger from the filing cabinet... millions of line errors!!!
Yes, the interface lead ran under the carpet just in front of that particular cabinet and that was the only time he went near it
(I've told this before...)
As have I with this one.
Early 1980s, minicomputer system in a general hospital, used for recording and storing pathology lab results (haematology, biochemistry, etc.). Customer complained of frequent crashes and reboots.
I happened to be on site one day, and was standing, with the customer, in the computer room, gazing blankly at the system. Then there was a "whomp!" which was felt rather than heard, and the lights momentarily dimmed by a lot.
"What was that?" I said.
"Oh, just the X-ray department next door".
Ah, many years ago we had a customer keeping some records on an Apple II system. They were suffering from corrupted floppy disks, but of course we could never find any problem with the hardware. So we surmised that it might be mains power quality issues as they ran some heavy duty spot welders in the factory. They tried moving the equipment to the office, but the problems still persisted.
Eventually we found the problem. The Apple literature of the time had pictures of an Apple II, with two floppy drives sat side by side on top, and then the monitor sat on top of those. Some will be ahead of me here ...
The Apple monitor had magnetic shielding in the bottom to allow this. The customer wasn't using an Apple monitor. Once they move the floppy drives away from the monitor, the problem disappeared.
Another famous (caveat: if true) case was where a particular hospital ward in the UK had a much higher than average rate of patients failing to respond to resuscitation measures. As it turned out someone was pulling the plug out so that they could clean and polish the floors.
The cleaner and the Intensive care unit is great urban myth I.e. unplugging the equipment to use a vacuum cleaner.
1. All ventilators have built in alarms for power failure and patient disconnection, pressure aberations, etc that arouse even the near dead.
2. ITUs work around the clock with a nurse by each bed and a supervisor overseeing the overall management.
3. Cleaning is lots of wiping and mopping...not vacuuming.
QED. [Long time Intensivist]
hospital ward in the UK had a much higher than average rate of patients failing to respond
Yeah I'm pretty sure this is an urban legend. I read a newspaper report about this (allegedly) happening in a hospital in Johannesburg, SA, but wasn't able to ever trace it anywhere. Doubt it ever happened.
The "news" article we are commenting on, already (or anyway now) has a link to "urban legend" i.e.
Also probably not true is the cleaner who spent all day cleaning an elevator car, since apparently they were under the impression it was a different one on each floor and so... but no.
Or the work experience kid!
One of my schoolfriends (a confirmed real human I personally know) got my school blacklisted by a local employer for work-experience placements after he unplugged a file server to make coffee.
In his defence, he was told to make coffee, and the kettle was sat next to the "server" for some reason. I mean, my mate was a dumbass, but the whole thing seems almost inevitable in hindsight.
Told to me by an oil company friend:
"We were posted to Egypt. While we were away a contractor came into our flat to do some work. They unplugged the freezer (full of meat) to use the socket for their drill and forgot to plug the freezer back in when they left. When we came back a week later we didn't even try and clean the freezer but had the whole thing with its putrid contents taken to the tip."
One place where I used to work, we had pretty much the same thing happen when they put a raised floor in the server room, and the chippy unplugged a server to plug in his drill. Then I tihnk the sparky they got in to put in redundant power circuits managed to do pretty much the same thing.
I'm sure there's an appropriate adage abut employing the cheapest tender and the quality of work you'll get...
I wouldn't. The IT manager in that place started off working in the warehouse (where he was known for sleeping in the racks), and "worked" his way up. The amount he knew about IT could be written on the back of a very small postage stamp, and still leave room for a detailed description of his common sense. The whole place was run by clowns. It changed its name twice in the four years I worked there, which is always something deeply suspicious*, and went out of business around six months after I jumped ship for a better job.
*I can think of two reasons for a company to change its name; firstly if it is successful and is bought out by a larger business and rebranded as part of its parent company, or secondly, if the company name has become toxic and the directors want to hide their past from potential customers. Guess which one applied here?
Two big flaws in that situation.
1. You let the monkeys work unsupervised.
2. You had regular power outlets.
You won't find anything but C13 sockets near the server racks in my Server room.
(Distribution rails, connected to large 32Amp sockets under the raised floor. )
"Bob's" story is so reminiscent of...
With torch shining bright he strode on in the night
'Til he came to the room with the safe.
"Hello son, I hope you're having fun."
"You've got it wrong, sir, I'm only the cleaner."
With that he fired, the other saying as he died,
"you've done me wrong." It's the same old song forever.
Nah, though that's what I guessed for this one.
I had a print server in a remote location that went offline and rebooted three times a day. That one was sited next to a coffee machine ...
I had another that went offline every night at around 7pm. Turned out that the socket was wired inline with the light switch per NY code ...
Then there was the time an entire building filled with data centers went offline because the building owner decided to remove a partition wall and the sawzall made short work of the wall and the forearm-sized bundle of optic fiber wiring inside it ...
And around the same time another building not far from that went dark, then caught fire when the electrician called in to install a circuit decided to drill a new cable access hole in the breaker box without pulling the main breaker, demonstrating just how much electricity is needed to set light to the fizzing corpse of an electrician ...
That reminds me ... We banned floor wax in data centers when HDDs first became common ... The reason was that the wax made it easier for the big machines to "walk" when in use, sometimes far enough to pull their own power and/or data cables. I had one block the only door in once, in the throws of committing suicide ... I had to climb over the hanging ceiling to get in to restore order.
At DEC and SLAC, we actually pulled the tiles a few at a time, took them outside & scuffed them with 120 grit on an orbital sander, wiped them down with a tack cloth and then replaced them. Made for better traction, and fewer incidents of equipment going walkies without permission.
This precedent of "telling the cleaning staff their job" (as it was claimed) made it easier when we finally kicked them (almost) completely out of the machine room ...
This reminds me of a time a colleague and I were setting up a Windows 2000 Domain for a further education college. We had a single 30U rack cabinet on wheels which had two DCs, a file server, some switches and a UPS.
The idea was that we could wheel this into any one of four IT classrooms and connect it up, and the students could learn about the joys of joining workstations to the domain, doing RIS deployments and all that lovely MCP goodness.
So we'd set up the DC's and everything, and were shutting the rack down, prior to disconnecting and moving it to another room. At the exact moment that my friend unplugged the UPS from the mains, the whole college suffered an unrelated power outage...
The look on his face, as he stood there with the plug in his hand whilst all the lights went out, was priceless.
Years ago my workplace had a satellite ground station that needed to run 7x24. The various bits were all plugged in to a wall-mounted power bar that was connected to a hefty Ferrups UPS. About once a week the UPS log would report an "overload". Checking the times of the shutdowns we found it correlated with the cleaning schedule. The regular outlets in the room were all under the equipment tables, so the UPS-supplied power bar was the most readily accessible place to plug in a vacuum cleaner. The cleaners were told to use outlets in the hallway outside the room.
I've posted this here before, but once in the 90s I got a phone call from a screaming client who demanded I get on-site now and threatening all sorts of legal retribution for work lost, downtime etc, etc, etc. So I drove the 3 hours to the site, walked up to the room had housed the computer and plugged it back into the wall. The shouty customer then got very quiet and begged me not to bill for my time or let head office become aware of what had happened.
I just remembered... more than one recent story on the (Customer Is) "Not Always Right" consists of a hotel guest, probably in 2019 or 2020 I admit, complaining that their room wasn't cleaned, and it is discovered that they had hung out the "Do Not Disturb" sign. Which is respected absolutely, in these stories anyway.
A "Do not enter" sign as described is not the same thing, and conceivably your cleaner doesn't read English. Even if they were born speaking it.
So really you need to display the familiar graphic of two pairs of feet thought provokingly combined. :-)
We had just bought first large Unix server and were in the process of preparing it for handover to the shift operators. It came with a menu system which gave access to all sysadmin functions. Shift ops had been given minimal training, just enough to take backups, start and stop the server and get the print spooler running so I wanted to modify the menu to restrict access to just these functions for them, My PHB didn't want me to put a sysadmin on the task as it would 'wasted days of an expensive resources time'. whilst you could add a new item or amend an item name easily, restricting access was going to require some real coding. to prove a point my sysadmin added an item called 'Don't press this button ever it will wipe the server'.
We briefed the ops, told them it would initiate a factory reset and the backups would be no good and left them to it.
We got through one day shift. 20 minutes into the evening shift it logged the fact that it had been activated.
Needless to say all the button did was log the event and tell the operator to inform us who had pressed the button.
When we grilled the two ops the following evening sure enough 'Bill' admitted that he had pressed it 'to see if it would really wipe the server' as he didn't believe we would leave something like that available.
Higher up the menu was a very piss poor recovery option which would assume there was a tape inserted in the drive with a tar file containing a backup. If he had pressed that option it would have copied the contents of the empty tape over the disk. I did then get a sysadmin to write a new menu so the ops could issue the commands they needed which were properly packaged, contained confirmation requests checked the state of the system before shutdowns etc. and we spent some time skilling up the ops on how different Unix was to the mainframes they were currently operating and how much less secure and resilient it was at the time.
There’s a photo (can’t be arsed to look up a link) of our illustrious Vice President Mike Pence touring some NASA facility (I think building the next Mars rover), lovingly stroking a piece of hardware prominently labeled CRITICAL FLIGHT HARDWARE - DO NOT TOUCH
The number of times I've seen someone click through warning boxes that would have explained the entir issue if they'd just read them...
Though I blame UI design for putting in too many warnings that you do click through on autopilot. I think a lot of cues can be taken from videogames with their "tutorial messages" thing - put the warnings up for newer users, but later on you're only seeing popups when it's something novel or important.
I blame business practice for setting up a system that requires users to do their work by clicking through a dialog that says "You do realise that the security certificate on the server expired in 2016".
Or indeed "I am your up to date web browser and I'm going to ask you 10 times whether you really want to run an Adobe Flash program even though you have to do it every day."
Yep, we had a plug with a DO NOT TOUCH sign on it in bright red, bold, underlined, plus the plug switch was taped down with yellow and black electrical tape. We were running 3D renders of a new building overnight (back in the day, when it would take all night). We had a super-urgent request for a drawing that had to be shown to a certain royal personage with an interest in architecture the next day. I started the render and left for the night. The cleaner came in, could't find a spare plug, so just pulled off the tape and plugged the vacuum cleaner in. Next morning panic ensued and the cost of a motorcycle courier to get the drawing to its destination in time for the crucial meeting was considerable. The whereabouts of the cleaner is unknown... ;-)
I have automatic garage door openers.
The delivery folks think "in front of the garage door" is an excellent place to leave packages
Half awake, get into car, hit door opener button, drive out...over package left in front of door
Create sign: "PACKAGES on porch, please. Thanks!"
Place sign between garage doors
90% of packages left on porch
10% still left in front of door, right next to large sign.
10% of delivery droids either: can't read English, are too lazy, or just DGAF?.
I once got home to find a card saying "we've put your parcel in the bin".
WTF???? What right is it of them to decide to throw away my parcel???
I look in the bin. No parcel.
I look in next door's bin. No parcel.
On the way back to the kitchen I notice the lid on the water butt askew. Yep. Ruined parcel of books bobbing about in the water butt.
I wrote a text adventure once with a large red "Emergency Reset Button" in it, which -- rather unsurprisingly, at least in hindsight -- transported you right back to the beginning of the game and wiped out all your progress.
If you spent too many moves in a row in that room without typing IGNORE BUTTON, your character pressed it anyway .....
Had a clown laying new vinyl in corridors including the one outside the computer room. Which contained a VAX and a couple of Novell (I'm that old) servers.
JUST stopped him unplugging the VAX to plug in some power tool - his hand was on the plug. I showed him a more appropriate power socket, but his extension cord wouldn't reach it. So while he waited for someone to fetch a longer one he lit a cigarette. Next to an open can of his adhesive, which was providing lots of volatile fumes. He was seriously pissed when I grabbed the cigarette and stubbed it out.
Had a clown laying new vinyl in corridors including the one outside the computer room. Which contained a VAX and a couple of Novell (I'm that old) servers.
JUST stopped him unplugging the VAX to plug in some power tool - his hand was on the plug. I showed him a more appropriate power socket, but his extension cord wouldn't reach it.
I'm confused is the Vax the server or the vacuum cleaner?
I think that upper case VAX probably implies the DEC VAX: https://en.wikipedia.org/wiki/VAX, rather than the vacuum cleaner from a company that happens to be called "Vax". I suppose anyone outside of the early 80's will not have had much cause to know about the old school VAX, as DEC's Alpha was all the rage in the 90's boom times.
is the Vax the server or the vacuum cleaner?
When I first saw a Vax vacuum cleaner I commented on the hubris of a manufacturer apparently naming a humble domestic appliance after the VAX. Twenty-five years on, and not only is the VAX no more (superseded by the Alpha, and who knows what since), but DEC has followed it into oblivion. Meanwhile, the Vax keeps sucking.
Ah 3 people who have never had to deal with stuff from the ex British now Singapore vacuum manufacturer.
Clogged cyclones, snapping clips, filters clogging and needing regular replacement.
Dyson products suck except their vacuum cleaners is pretty accurate.
Heard about someone (at CERN?) who was tasked with doing something in a highly radioactive room. He was provided with a protective suit and thoroughly trained in the task. About 4 hours into the task he stopped, opened up his face mask, lit a cigarette and started smoking it. Seems he was on break.
The janitorial staff's job is to clean the place. Floor to ceiling, board room to bog, watering plants, replacing dead light bulbs & emptying the trash in their wake. The modern world wouldn't run without janitorial staff.
But leaving the computers accessible to all and sundry is a major security fail. So why the fuck didn't whoever was in charge see to it that that one particular door had a non-standard key for the duration? All institutional locking systems that I am aware of have a provision for making specific doors unaccessible via the standard staff master key.
I'm guessing that, given that it was in a hotel, the equipment was regarded as about the same as the vacuum cleaner. Even though the hotel would not function if the server didn't, it still didn't twig anybody to treat it with care.
I'm wouldn't be surprised to learn that hotel receptionists today would do practically the same.
“ I'm wouldn't be surprised to learn that hotel receptionists today would do practically the same.”
My wife works as a housekeeper in a well known budget hotel chain.
And the answer is yes, they absolutely would, because they’re working to inflexible checklists, schedules, and requirements which say that certain very specific things have to happen in certain very specific places within certain very specific timeframes. If they can’t show that those things have happened then the consequences range from loss of bonus payments (which while derisory are still a big deal when you’re dealing with the pay scales at the bottom end of the hospitality industry food chain) to disciplinary proceedings.
So yes, if they haven’t got specific written instructions from local management, signed off by the people at head office who audit and compliance check the time sheets, daily reports etc then damn right they’re going to unplug your server to do the hoovering,,,
"The modern world wouldn't run without janitorial staff." Too true.
A few years ago, at a previous company, we discovered who the most business critical staff-member was the hard way. We could do without a CEO for a few weeks, even a head of IT (sorry). But the day when the below-minimum-wage cleaner who replaced the toilet roll in the loos failed to turn up? Immediate chaos. Having the CTO ring up to get someone to take a roll into the toilets where he was stuck remains one of the funniest moments I've had at work.
Please don't be mean to cleaners.
Douglas Adams nailed that one...
The Golgafrinchan Ark Fleet Ship B was a way of removing the basically useless citizens from the planet of Golgafrincham. The ship was filled with all the middlemen of Golgafrincham, such as the telephone sanitisers, account executives, hairdressers, tired TV producers, insurance salesmen, personnel officers, security guards, public relations executives, and management consultants.
A notation in the Guide about Golgafrincham after the departure of the B Ark states that the entire remaining population subsequently died from a virulent disease contracted from a dirty telephone.
Ironically, just before the great plague descended on us, I ordered a (Reasonably specced & priced) Android 10 phone from China.
After 2 months the thing has finally shipped due to various reasons (Out of stock &\or Chinese New Year, Coronavirus* & got a discount\refund when I complained) in time for the great flight shutdown.
*Which is potentially being included with the phone, as a courtsey detaill.
Icon - PH knows a thing or two about various nasties.
Because when you have the likes of Pullman St Pancras, a big chain hotel, who can't even be bothered to wireless isolate their whole network, allow you to wondering around looking at what other devices are on the network. And have server IIS installs open to anyone wondering round on their un-isolated network you can see even the big chains don't give a shit about IT security.
To be fair, as I was in my room reporting it to them via Twitter, that very day they started to lock it all down.
This post has been deleted by a moderator
I work at a very large semiconductor company in the PNW. *Incredibly* large fab. I make sure to smile, nod, and/or thank the cafe and cleaning staff at every eye contact; I try to make eye contact. Especially the ones that clean the clean room!
A/C, well, because.
Except when it's not.
Had an engineering study set up to run overnight. Cabinet locked. Sign posted on cabinet. Three bands of tape above on and below latch. Cart of wired up equipment parked in front.
Next morning cart is to the side.
Someones project inside cabinet.
Three temperature spikes logged.
Two were on the downswing so still in spec. Third was at the peak and out of spec.
Some of the sensors had been moved.
Study fail and has to be repeated.
Sr researcher and board member walks in and wants his samples. Turns out he was using some of the just delivered bench instruments as well.
He was not happy that his budget had to pay for the repeat of the study and also to explain to the ministry auditor the reason for the delay.
This is where phd becomes Fudd as in Elmer.
He was not happy that his budget had to pay for the repeat of the study and also to explain to the ministry auditor the reason for the delay.
Unless it is a long time ago and that researcher/BM is long retired, I strongly recommend you guard your back carefully, those senior idiots have a tendency to hold a grudge and will try to retaliate.
I still have a Win2K box up and running. It's the only Redmond OS I still have running. I use it nearly every day. The only reason she still exists is to run ACad2K. She has never, not once, been down when I needed her. The only time I remember her crashing was when I was sorting out drivers 20 years ago. Win2K was the absolute peak of Redmond operating systems.
She's airgapped now, so fuhgeddaboudit.
Being a janitor, I know better than to ever unplug any computer - desktop or otherwise. Granted, 99% of the places I've cleaned don't allow us anywhere near the servers, but I'd still never unplug anything. Even using the same outlet as a client's gear is a no-no.
That having been said, I once took over a site from a lovely older gentleman. My employer informed me that he was being removed because he'd received a complaint about the dusting of the desks. The problem wasn't the complaint itself, but the fact that he disconnected and moved everyone's PCs in order to do a very thorough dusting. Didn't shut them down first, just yanked the power cords and everything else that plugged into the back of the tower.
Poor old guy genuinely didn't understand why he was being taken off that contract.
Beer, because I desperately needed one after finding all that out.
My work place forces computers and all electric stuff shutdown before leaving for the day. It's mandatory. Shit happens when you didn't, you get the shaft.
THough My office is inside a rafinery (previously inside a similar plant). So all electric stuff must be watched or closed as they are seen as fire hazard.
I had that habit for anywhere one. Shut down the computer for the night if not needed.
Refineries have maths-head engineers sit down and calculate a Mortality Gradient for every identified fire/explosion/etc risk.
The gradient allows a quick visual understanding of the number of fatalities at each location, office, etc for each event.
So-oooo... I am in full sympathy with a policy mandating shutting off all unused elec kit! :)
Working for a well-known (mostly? almost? not sure now but I think it still was back then!) British company in the 90s and we started getting rather angry calls from our Users complaining that some of their computers wouldn't reboot.
We had a couple of our on-site hardware engineers (the youngest pair, let's call them "Tweedledum" and "Tweedledee") that had been PAT-trained as none of the other (older, more experienced) engineers wanted to do it.
"That's okay," says the Boss, "We have a couple of hardware engineers over there doing PAT testing, we'll get them to pop up and take a look."
Some of you will have noticed I said "reboot" and "PAT testing" and worked out what had gone wrong. Or, rather, who. Sure enough, when the Boss tracked down The Thickie Twins it turned out it hadn't actually occurred to them that they should shut down the computers properly before yanking out the mains leads and connecting up the PAT tester... or even, on some of the desks, to actually plug the things back in before walking off to the next one...
(On a slightly related note, Site Services had done the testing for stuff like kettles, desk lamps etc - Tweedledum and Tweedeledee were "just" doing all the IT kit. Except they forgot that included things like printers, scanners, routers/switches... Strangely, a couple of the other engineers got trained up to do the 'catch-up' stuff. I sometimes wonder if it might have been one of the 'customer' team managers threatening bodily dismemberment for Tweedledum and Tweedledee, and whoever sent them back over that inspired that..?)
Years ago we had some guys wandering round the country installing comms gear that drew a hefty current when first powered up, so they were told they had to check the power with an earth leakage tester before they could install the kit.
They installed at a few sites and had one or two sites where that had to say "sorry mate, you need to get a sparks in first"... then they had a site where the kit was installed in a data centre.
The tester was plugged in, the test button was pushed, which shot a spike down the earth wire... and a hush descended on the noisy room
Funny thing is the requirement for an earth leakage test was discontinued after that...
The cleaners here won't even clean the desks because they have computers on them. I've got an interesting experiment going on to see how thick a layer of dust I can build up over the course of a few years*
* technically it's a control group, because I have a similar ongoing experiment at home, which is down to pure laziness on my part.
Yes, at my last company the cleaners did not go into the lab areas (even after a fly infestation!)
Back to the late 1980's. A husband and wife translation team worked from home. The husband’s computer kept losing the Wordstar (I think) documents, but never when he was at the desk. I soon had him saving the work to floppy on a regular basis, but was despatched by my boss to see what was happening. I had a very boring morning watching this couple in their living room. The husband went to the shops at lunchtime and the wife looked to me and put her finger to her lips. She then proceeded to remove the plug from his monitor, plug it into the printer and print off a couple of documents, before replacing the plug in his monitor. Unlike her computer with a separate CPU box, his was integrated and of course her removing the plug removed the power to the CPU as well. I had set up the autoexec.bat (or whatever it was then) to run Wordstar when it was powered on, so the husband only saw a loss of the document. Once I had explained the issue to the wife, I left a fault sheet listing a power cable entanglement issue (i.e. something that was their fault and I would not be blamed for) and then left. I never had another call from them, other than a regular order for more printer ribbons.
A few days ago everyone in the office where I work received an email asking them to clear their desks because the cleaners would be using a special anti-coronavirus spray on them. The odds and ends at the back of my desk had been there for some time, so removing them exposed a thick layer of dust.
I can report that the cleaners have apparently managed to apply the magic prophylactic spray every day this week without disturbing the dust.
So it's not always the cleaners that do it.
Read the specs. The noise is battery powered, the mains keeps the battery topped up.
A small gelcell, sonalert+LED and relay does pretty much the same thing. A simple charging circuit keeps the gelcell topped up. The battery will power a sonalert for a surprisingly long time. Most datacenter cabinets should have them mounted ... although many have the noise unplugged.
These things have lots of similar uses ... At various times, I've had 'em mounted on my office door, file cabinet, desk drawers, the doors of various equipment racks, tool boxen, etc.
At a certain company of my acquaintance, they were building a brand new server room. What's more, there was actually an appropriate budget for new equipment (mirabile dictu)!
So, on our advice, they bought several UPS's, with the capacity to run the expected load should the racks ever be filled. However, as the new server room was in a separate country, we left it to them to install everything themselves.
A few months afterwards, they suffered a power cut, but the UPS's didn't seem to have worked, they had shut down at the same time, and everything had lost power. Eventually we found the problem, the (very stable) genius who plugged in all the devices, had plugged UPS1 into the wall, then plugged UPS2 in to UPS1 and so on. When the power had cut, each UPS along the chain had been drawing power from the next UPS along, which left poor UPS1 with the entire power draw of the server room, whereupon it shutdown, UPS2 did the same, and so on. Only the UPS at the end of the chain (which was also the least loaded) survived, and pretty much everything else went off.
Fortunately the UPS's had shutdown without damage, so with some new kettle leads to give them a mains supply, order and redundancy were restored.
(anon for obvious reasons)
I had something similar... a comms rack on a temporary mains feed (long lead with rubber socket on the end that disappeared through a hole in the wall into the next room)... 2nd rack (servers) then added, plugged into 1st... 3rd rack (switching gear) plugged into the 2nd... 4th rack (thankfully empty) plugged into the 3rd
The rubber socket on the end of the trailing lead was a little warm... warm enough to have burnt a mark on the (thankfully) stone floor
Have a client who is currently insisting on being able to connect their equipment that way (minimizing number of outlets used, don't you know).
I have warned them that the Electrical Inspector may find that objectionable. They have held a meeting and have informed me that they do not belive that will be the case and that they can handle it, and I should focus my efforts elsewhere.
Had similar. Cleaners dusting the top of the portable AC units we had been given to cool the server room in the back of the IT office. Said AC units had touch-sensitive controls, and the dusting switched them off.
We got proper AC installed on the back of this, and a lock on the server room door.
I'm trying to come up with some way you could make a plug that can't be removed without a key.
The easy solution would involve a modified faceplate, one that adds a hasp next to the outlet and a projecting loop next to the plug. Put in the plug, add a lock, and you're good to go. (Unless the cleaner brings along a screwdriver to undo the faceplate... I'll ignore that issue, and assume we just want to stop someone from casually pulling the plug.)
Ideally, I'd want a gadget suitable for adding to existing equipment and outlet : turn off equipment and unplug it, plug gadget into wall, plug device into gadget, turn key to lock it in place, turn equipment back on, and put key into pocket.
It'd be easy enough to have the gadget hold the plug in place. It's keeping the gadget locked to the outlet that poses a larger engineering challenge.
All you need is a locking plug cover. I've used these in the past with good results. The smallest of locks will stop all but folks who are determined to be disruptive ... and they can bring cutters to the party, so unless you want to invest in armored cables ... anyway, use your favorite search engine, search on "lockable plug cover" or "locking Receptacle Cover".
 Locks are only there to stop crimes of opportunity. Most are easily bypassed.
> I'm trying to come up with some way you could make a plug that can't be removed without a key.
They're all over the place in big shopping centres over here in oz.
Typically just a clear perspex box projecting around the sockets, perhaps 4"/10cm deep to allow lots of room inside for plug, holes _just_ big enough for the cord to slot in, and the opening lid fitted with a lock.
Transparent so that it's immediately obvious that it's just a powersocket, plus is easy to find/recognise. Yet keeps kids' sticky fingers etc out of it.
On the one hand, having an electric plug that you can't unplug seems like a breach of safety. On the other hand, there are installations where an electricity cord just goes through a hole in the plastic face of what looks like an electric socket, presumably isn't a socket strictly but has similar stuff inside i.e. wires with electricity in them. Nothing to unplug, but you could pull really hard... if switching off doesn't work. Perhaps there are rather strict rules about what can be on the undiscussed end of the cord.
The trouble really started when they named the software. Property Management System (PMS)?!?!
On a somewhat unrelated note, it reminds me of a company I worked for that went though yet another cycle of renaming department because the names weren't cool anymore.
The term "Team" had been deemed uncool and was to be replaced with "Services". So:
Application Team (AT) -> Application Service (AT)
Project Management Team -> Project Management Services (PMS)
It wasn't long before we noticed that the gal in the cubicle next to us answered the phone with:
"This is Susie - thanks for calling PMS, how can I help you?"
Whenever we were in the pits over something we would wait for that phone to ring...
We even contemplated calling her ourselves, but figured that would eventually get us in trouble.
Sometimes it's the JCB operator.
I couldn't believe how much of a caricature this experience was. The office was an old mill building, next to a river. One morning, all comms went down, everything dead. I knew we had all our eggs in one basket as we the office was supposed to be temporary, and as everything was out, the issue could only have one origin. As I got up, I looked out of the window across the river, where I saw a JCB, bucket in the air, with the ripped innards of a thick cable hanging on either side. I almost laughed at the textbook example.
A quick trip across the river, to let them know the severity of their teeny weeny little bit of enthusiasm and several hours later all was well.
Bit like kleenex and tautliner, but with additional cultural specificity. It's a brit thing. As is being an exception ;)
JCB tends to imply council work or big construction work. And are big and yellow and have a special place in every tiny child's heart. They are Propah!Diggers. I dare say no Case 580n backhoe song ever tore up the UK music charts & t'internet, for example?
Whereas... the JCB has its own Song which did.
Down here south of Nashville, there is a church that has an annual "Builder" day for the kids. Their huge parking lot is filled with construction equipment. Best to avoid the traffic in the vicinity on that day.
It appears it's easier to put seats in dump trucks than it is in pews.
Joseph Cyril Bamford (ie the bloke who started the company that now bears his initials) was the first person to combine a rear excavator (or 'backhoe' as the yanks call it) with a front digger, to create what British people just call a 'JCB', regardless of manufacturer.
Shockingly, the Mk. 1 Excavator was actually red and blue, not the iconic yellow.
My first experience with unreliable power was at a small office move where they added a few outlets to accommodate the Novell server, phone system, and AT&T Unix voicemail server and associated network equipment. I was trying to keep related equipment on related circuits so a circuit outage would only affect one service instead of everything. It was not quite possible to do it that cleanly, the network switch had to be on the telecom circuit. I was seeing connectivity issues all over. Troubleshooting was a bear. I also had new Cat3 drops to suspect. I for some reason checked the grounds between the circuits for potential. The multimeter showed about 50VAC between the telecom circuit and the computer network circuit. The electrician confirmed and corrected the grounding issue, apparently there were multiple grounds for the building but they were not bonded. Connection issues disappeared.
At the same place one of the user's monitors (CRT) had an annoying "waviness" which was also suspected of causing headaches. When I saw the "waviness", I could see the possibility of headaches. I found the building's electrical feed was directly opposite the wall of the problem monitor. I first suggested rearranging the office to move the monitor, which was declined. I then attempted to shield with a large steel panel both in grounded and ungrounded states. The grounded steel panel diminished the waviness greatly but not completely. They decided to rearrange.
At another job there was a "computer lab" setup in a former cubicle farm space. Fifty or so stations were setup in a very long and narrow space with a server located halfway. This location had better than average policies which included periodically testing the building UPS systems. On the first test after full build out of the lab, half the lab went down, including the server. I had to inform project managers, facilities dept. and others that half the lab was not on the building UPS. The facilities dept. had to admit they knew that but the cost to add was prohibitive. The consensus decision driven from higher up was not to fix the issue. I moved the server to the half that was on the UPS.
At that location I was once volunteered as the escort for the UPS tech for a repair of one of the units. I asked what he was fixing. He replied, replacing the positive battery buss. I walked back across the room to the doorway. I said I wanted to be able to summon help in case of a short circuit. The battery cabinet footprint was about 3ft by 15ft, probably enough energy to blow me through the doorway and he would have been beyond help.
Once worked for a place in the UK that will remain nameless. The building/campus had an exceptionally large UPS in the cellar (it was the size of a bus). The UPS powered every computer in the entire building, including client workstations, through a bunch of red 13A sockets with "clean power" printed on them.
We had a cold and dark winter one year that resulted in many issues with building power. It was eventually discovered that various office workers were plugging their 3kW fan heaters into the clean supply, and the poor UPS was having to resort to rolling blackouts to try and cope with the load. No amount of training, polite or otherwise, was sufficient to break this habit: we ended up having to swap all the sockets over to ones with a specially shaped earth pin to prevent a regular plug from fitting in.
Linux server, back room of store, server not responding. Asked staff to go and make sure it was on. Staff came back, it is on. Now we could see server. Week later, same scenario. Called staff, is that server on in back room? Yep on return. It responded again. Scratching the grey. Next week, same scenario. Ghost of the back room server on same day, different times, and before the store opened. We asked the question, does anyone go into the room on that morning near the times stated. Yes was the response, the cleaner goes in. Then it all started coming together. We asked the staff to mark the socket plugs not to be unplugged. It never happened again. I wonder why! Oh and before you comment. No UPS was in line. I know, its stupid.
I remember making my holiday plans and training somebody to do the backups for me daily. All went well , Good holiday, When greece still had the drachma. When I got back I wondered where all the daily tapes had gone ... They hadnt been using a different tape every day, just the same one. Was very nervous for two more weeks till the cycle got back in sync. Worked next a substation and it never caused a problem neither.
When Skype was in its infancy - and still owned by cool dudes in shorts and flip-flops - I was responsible for setting up their card processing into Natwest via the behemoth that was Streamline. X.25 packet switched lines, make and break connections, APACS Standard 70, EBCDIC and RAS mainframes for batching and confirmation...those were the days.
Shortly after midnight, the Skype Settlement Service would contentedly roll up and batch transactions, dial into the RAS servers, update routing tables and pop the file into the merchant FTP directory and disconnect. Then dial back in a little while later and parse the confirmation file to make sure everything tallied.
All ran remarkably smoothly for a year. Volumes grew. Complacency set in. One blustery Sunday night, SMS alerts woke me from my slumber. Primary ISDN modem was unresponsive. Secondary modem (in the same location of course!) was also nowhere to be found. After the initial panic subsided, the folks on call in the data centre were dispatched to go and investigate.
Suffice to say, a cleaner was found with vacuum in hand and ISDN modems unplugged. The sullen offender was given a good bollocking by the then project manager. The poor chap wasn’t impressed and clearly hatched a plan for revenge…
A couple of nights later, SMS alerts came through again. This time, there was no sign of the cleaner…or the modems!
Quite a few rears ago, the car parking in my local railway station was operated from a portakabin in the car park. There was a power socket on the wall in the customer area, with a plug in it, the cable from which disappeared through a small hole in the wall into another room. Above the socket it was written "Do not unplug this EVER" or something very similar. On a couple of occasions I was very tempted to turn it off, partly to see what would happen, but mostly to get my own back, since on those occasions the attendants had managed to royally piss me off.
Yes, the Red Dwarf joke about car park attendants IS true, well it was true where those two clowns were concerned anyway!
I installed an ERP system into a small manufacturing company roughly a 3 hour journey from our office. Initially everything performed perfectly, but a few weeks later the customer complained of extremely sluggish performance. Dialling in (this was in the days of 33k modems - if you were lucky!) everything looked fine. All performing as expected. But the customer kept ringing back with the same complaint. Eventually I was dispatched to take a look on-site.
It didn't take long to find the problem; the server was running a very complicated screensaver that took up almost 100% of cpu time. Of course when I'd dialled in previously, the screensaver was disabled. Apparently the MD had let his young son play on the machine, and that was the result. Cost him a day of my time plus expenses.
It's one of the first things I check since then, but I've not come across it again since.
Biting the hand that feeds IT © 1998–2021