If we plug this in without telling anyone, nobody will know we caused the outage What? Monday? Again? Didn't we do Monday last week? OK, fine. Welcome once again to your soft landing pad into the working week – the oasis we call Who, Me? in which Reg readers unburden themselves with tales of tech tribulations. This week, meet a reader we'll Regomize as "Padraic" who spent some time in the 1990s working in …
They didn't bother to clip the SCSI connector in, and couldn't detect that a drive/device was disconnected during who knows how long a period of frantic troubleshooting, so I'd only give it 50/50 odds at best that there was a server room access log.
Or the access logs were themselves saved on the downed storage device ;)
This story reminds me of the old saying "SCSI isn't magic. There are fundamental technical reasons why it's necessary to sacrifice a young goat to your array every new moon"
My recommendations of a ritual sacrifice have never been approved.
On projects which become 'awkward' I undertake a procedure where mantras are invoked to aid the excision of demons. Those nearby think I am just muttering..... I leave a very small text file to maintain protection and also make an appropriate libation ---->
In the 1960's at Dartmouth we were running time-sharing on some GE computers. Every so often something would fail and the technician would run diagnostics and find nothing wrong. Bringing up the system we couldn't find anything wrong either. The technician caught on and ran diagnostics every morning. When asked, he said he was warding off evil spirits. Reliability was much improved.
One day his diagnostics failed but time-sharing ran perfectly. Further checking revealed the failure was in the "bit change zero" instruction that converted EBCDIC to friden flexowriter code or something like that. We couldn't figure out a use for this instruction so we didn't use it.
Back in the early 1990s, I worked on a US government contract. The server room access log was ring binder with forms in which you might write your time of arrival, your task, and your time of exit--office buildings commonly have such at the front desk. The boss contractor mentioned as an additional reason for their use that they could get one out of trouble: somebody accused of smoking pot in a stairwell was shown as have been in a server room at the time of the alleged offense. I don't think that I was particularly conscientious about filling them out.
The non electric multimode analogue data system (pen/cil and paper) is still used.
Last time i experienced the pure paper variant was in a railway maintenance facility in Solna (near Malmö/sweden) in 2019.
The main reason for the use of paper was that if there was a fire or power outage, any kind of emergency where you need to make sure everyone including contractors is out of the buildings, a computer system that might be incapacitated by said emergency wont help much.
Airbus, Mercedes and Volkswagen are using electronic access logs, the only pen and paper play is the signing of that ridiculous GDPR paper where Airbus is giving out pens with digitizer cameras to capture your signature.
Metal working facilities and chemical plants also tend to use paper alongside electronic access logging, so it seems that any plant where something really big and nasty might happen does not like to rely purely on electronic victim records.
server room?
My first one in the late 90s was at the back of the stationary cupboard, key at reception. It was an improvement because when I arrived we had an AS400 dumped on the service dept floor, imagine how many toolboxes etc were rested on it.
In the early 90s Very few of the server rooms I visited in London had logged key card access, only the big market makers, traders and technology companies. Enron oddly enough had one!
The managed service provider (third-party IT for SMBs) I started with in 2012 had their one small server hanging from industrial hooks in the bathroom. It did get moved to a shared rack in a closet with a co-tenant in another building when we moved at the end of that year.
Even now, you’ll be surprised of how many companies simply have no concept of the importance of their servers and other network infrastructure and simply stick them wherever. Even when a purpose designed ‘room’ is set aside for servers, they have barely settled in before said room becomes a general storage facility for stationery, bicycles (don’t ask), etc.
There was one client of mine some many years ago who had their servers and network kit in a rack in a nice air conditioned room, but they were a magazine publisher and as such often got sent ‘samples’ of various products for review. One day we found the server rack entirely surrounded with boxes and boxes of perfumes and deodorant sprays. I’ll always remember my colleague at the time telling the office manager that ‘if they ever have a fire, they’ll find bits of your fucking servers on the moon!’
Once, once, I managed to get it done properly. An advertising/marketing company were moving into a complete new build facility and I managed to get on the design team. I wasn’t interested in the general office payout, but they had reserved a space for ‘the server room’. I managed to get this space divided into two, one part was just big enough to get the four 42U racks in, with sufficient space front and back to install equipment. The other part was much larger and was intended as a ‘build area’ / storage area for IT. Guess what happened to the outer room?
The inner one, being far too small and narrow to put much else in, got left alone. Plus I kept changing the combination on the door lock, so it didn’t take too long for everyone to get the message.
This is why some of these Who Me? stories, while being very amusing, I take with a pinch of salt. Any sysop faced with this situation is immediately going to identify the guys working in the server room as suspects number one, and go check the hardware.
Only if you suspect the hardware.
I worked at a place where one day every wax in panic because continuous builds didn’t build. It took two hours trying to get it working until someone decided to go to the server room.
They found a window broken open, and no server.
That's the problem with remote servers, you stop thinking of them as hardware in the first place. Even worse is "cloud" stuff where everyone remembers the decade old sales pitch of virtual machines being seamlessly passed between servers without ever impacting client experience, yet, inevitably, your "cloud application" has been running in a single unchanging rack in a single warehouse somewhere in Dublin for the last six months, and won't move without an outage.
Because cloud! It saves money because we don't have to pay techies! And we don't know where our data's physically hosted so we can use that as an excuse on data protection enquiries*! Everybody wins!
* didn't Facebook actually do this once, I seem to remember?
“ This is why some of these Who Me? stories, while being very amusing, I take with a pinch of salt. Any sysop faced with this situation is immediately going to identify the guys working in the server room as suspects number one, and go check the hardware.”
True, except that assumes that a) the Sysop is competent, and b) is actually aware that someone has gone into the server room.
Although, in theory the sysop, or whoever has that title, ‘should’ be aware of what is going on, in reality, often not so much. The server room / general stationary store / coat drying room / bicycle shed, might have more traffic going in and out than a major railway station!
Ah the 10p saving that ends up costing thousands – classic. If you've ever found yourself in a mess that could have been prevented by some spare change, let us know about it
Buts lets exclude 10p invested in a firearm round. We all have stories that could have been best avoided by that simple investment.
Canadians have pennies!
The Royal Canadian Mint stopped producing and distributing pennies in Canada as of February 4, 2013 due to rising costs relative to face value and the significant handling costs of the penny for retailers, financial institutions and the economy in general. Reference Canadian Bankers Association.
"We all have stories that could have been best avoided by that simple investment."
WTF?
I don't. I'd bet that there are many people living in civilised countries who haven't had any need in their lives to escalate any problem to one involving lethal force.
The OG SCSI-1 Centronics style connectors with the clips were fairly secure (if they were properly clipped) but they went away when SCSI went wide. So by the 90s those had fallen by the wayside for wide/ultrawide SCSI using the high density connector which was a nightmare!
I can't count the number of times that intermittent problems could be traced to a dodgy 68 pin UWSCSI cable that had one pin that was slightly bent or had been pushed in a bit, so that that it didn't make solid contact. It could be properly connected and have the screw things on the sides actually screwed in (which were another problem altogether...) but still be in a situation where wiggling the connector or pushing it hard up or down or in could make an intermittent problem go away. But heat cycles or the slightest bump of the cable would undo that magic and it would again become intermittent. The only real fix was to trash the cable in favor of a new one, and hope that once it is plugged in it is never moved - because repeated plug/remove cycles were what created the bent/pushed in pins in the first place!!
I'm glad I got into storage consulting when fibre channel was going mainstream, so I only had one contract where I ever had to deal with servers connected to a Symmetrix with SCSI! Unfortunately JBODSs were still attached with SCSI so you still had to deal with those terribly designed 68 pin HD connectors for server boot drives/scratch storage.
I'll see your 120 pin EDAC and raise you the complete back plane of a VAX 11/750, when the massbus started playing up. Me (sysop for the VAX) and a Cray engineer (very bright guy) spent a happy early morning with a pulse mode oscilloscope trying to figure out why the VAX occasionally crashed the Cray it was connected to. Yup, two bent pins on the backplane. Feeling of achievement when we tweaked a pin by less than a mm and everything was rosy,
Mines the one with the HP Oscilloscope manual in the pocket.
This is what I love SCA connectors for. One plug, mechanically much better, done.
Never ever ever had a problem with SCSI. Ever. Possibly because we insisted on using high quality full wired and shielded cables.
And I wonder if the fundamental reason was really "missing a connector" as cables came with connectors as standard
- 50pin SCS-2 centroncs interface on the equiomennt had two clips to lock onto the cable.
The female version had the same built onto the cable
- 68pin SCSI-3 *aka Wide-SCSI" had a spring latch built into the cable connector
-DB25 abortion used a standard screw connector
So... no clips, 10p or otherwise. So either
- someone hadnt fitted correctly
- something had been broken and nor replaced
- someone disconnected a cable..............
I assumed the article was talking about the Centronics style with the clips and someone had just neglected to engage the clips. That was often ignored, just like the "screws" on the side of the DB25 style or 68 pin UWSCSI cables were ignored (the ones with spring latches were worse than the ones with the screws assuming the screws were properly engaged which was its own problem given that depending on location you might not have room for a very long screwdriver or any vision of what you're trying to screw)
At my last job, we had to be very, very careful when working on one of our racks, because one of the fibres didn't quite fit soundly in the back of one of the servers.
We all knew this, so we never had a problem.
Well, we didn't until the aircon engineer visited.
The aircon engineer who dropped his screwdriver.
The screwdriver that just happened to graze that exact fibre as it fell to earth.
Cue lots of frantic teachers calling because they'd lost access to all the vital powerpoints held on that server...
I think I read that one of ethernet's inventors (Bob Metcalfe?) greatest regret was the aui connector - the retaining sliders I think.
The fairly heavy media adaptors were always falling of the back of machines.
Fortunately once twisted pair became de rigeur the e-waste box of old cards and adaptors filled quickly. :)
Fixed with so called "snagless" hoods that prevented the tab catching on other cables as you tried to untangle the knitting at the back of switches and servers.
Of course, sometimes these prevented the connectors fitting into recessed ports on the back of some servers.
I hated the AUI slide on older systems. Where ports were close together, you needed a screwdriver or some other stiff, thin object to get to the slide to undo it. I'm sure that some of them 'fell' off because someone couldn't get their fingers in, and just pulled.
In case you had forgotten, the cables that came out of the AUI were very heavy. They needed a quite substantial slide clip to prevent the weight of the cable pulling them out of the back of the system!
Ah, but the snagless ethernet cables came with their own hazards.
I was at one of the IEEE meetings where AMP made a pitch for the slide latch.
What a mistake. I hated those damn things. I have ONE memento of that era...an old SGI Indigo with an AUI connector on the back and an AUI to 10BASE-T adapter so I can hook it to my wired network.
Once upon a time I worked at a place where there was an Apollo Domain ring of workstations.
For some reason the powers be that had fitted some external SCSI discs storing general user files to a slightly higher end tower form factor machine.
It was sitting under a table in a general office space, and I stretched my legs out and kicked the cable..
I was sitting at the machine because my simulations ran faster on that machine.
Gradually some peoples files stopped saving .. as their machines network caches filled up as files ceased to be written any more..
I've seen someone get a DIMM in the wrong way around, but not deterred by the notch being in the wrong place, they managed to jam it in hard enough to engage the latch on both ends.
Kind of impressive in it's own way. IIRC it worked fine once the DIMM had been re-inserted correctly.
Not inherently bad. Just certain kinds. The 75 ohm ones used in CCTV, mostly legacy now, I think. The modern compression type are pretty good. But I run into the older hex-crimp ones that were applied with the wrong tooling. They go bad (or were bad, just not detected). And there's a special corner in a very hot place reserved for anyone who thought that making, or using, the ones that don't crimp at all, they just have a female screw thread to capture the shield. I find those causing problems a lot. I can imagine the marketing speil: Easy to apply! Saves time! No tooling! No crimping! Just twist on and go! I think I remember seeing them in Radio Shack blister packs.
Honorable mention: the DVR makers who jam the BNCs so close together that you (or at least I) can't get fingers between them to do up, or undo, the retention rings. I find them unlatched, and the plug having worked its way out, sometimes.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2025
