Software guy smashes through the Somebody Else's Problem field to save the day A warning from the past in today's On Call. Helpfulness is not always rewarded with a pat on the back and a slap-up meal on expenses. Sometimes the Somebody Else's Problem field* is best left alone. Our tale comes from a reader Regomised as Derek and concerns his time working for a multinational with plants at multiple …
I've told this before, did not get a stern talking but also could not correct the problem.
During the Y2K project I was in, my team was correcting an application consisting of almost a 100 programs chained together (just as COBOL likes) and we were made aware that that particular batch chain would break every single night, meaning an hefty on-call bill. And is had been going like that for years.
So I analysed the failure point and it was seemingly simple to correct - IIRC there were numeric conversions that would fail under certain conditions, conditions that would almost certainly be met once or twice per run.
That correction was deemed out-of-scope by our client rep, who, curiously, was also that application's IT boss.
I was possibly standing outside when the story teller was getting the dressing down (it certainly sounds very similar to the story I could hear through the door!)
I was awaiting my turn for fixing the "application error" by putting paper in the printer - again not our job! I was told the unions would be up in arms, but I heard nothing more.
I was tech support manager for a County Council. When the legislation on working on electrical equipment changed I had to organise a course on how to fit a plug on a mains lead for my comms team. The same team who literally built the x.25 switches for each site at the time from the chassis up :(.
Our site electrician was actually embarrassed to have to do this for us but we needed the sign off of the H&SE ream were going to ban us from working on any electrical equipment. The Desktop team didn't have to have the training as all their mains leads came with molded plugs and were never re-wired.
By the nature of the role members of both teams would regularly be working on hot devices with the case removed when performing testing.
Indeed I was contracting at Motorola nd they had an on-site course on programming security techniques.
At the beginning the instructor went around the room asking each employee why they are there, they were saying good stuff like "to improve my knowledge of how to program with less vulnerabilities" and the like.
Then he got to me and I simply replied "I'm a contractor, I'm paid to be here". At which all the other permie engineers burst out laughing :-)
Same - I had to sit through a cyber security video before going on site. Notwithstanding that I had personally designed, written and recorded said video as a consultant to the site owner. I took the liberty to watch the thing in a language entirely foreign to me (translated by one of my native-speaker colleagues from my English version), just for fun.
This is kindergarden school level. Fellow how worked with my father, long ago, was forbidden to have a copy of, or read from a paper he had written. My father however could have, a copy of said paper in Toronto or Houston without problem, since he had a security clearance (born in UK) while the writer of the paper could not qualify having been born in what is now part of Hungary, but was then (the horror) Russia!
True.
But the generally accepted definition of nationhood is membership of the UN. Even when they were controlled by Moscow Hungary and the other Warsaw Pact countries were members of the UN. Apart from Ukraine and Belarus, former members of the Soviet Union - Estonia, the 'stans, Georgia, etc - weren't UN members. They only got recognition as sovereign states when they declared independence after the SU collapsed.
I wrote a strategy paper for a client but wasn't allowed to present it to the department concerned as I was a contractor.
I was also told I couldn't go to the all-hands session where it was being presented as it was company confidential (again, just for clarify - I wrote the thing in the first place).
I spent a delightful afternoon sitting in an empty department drinking tea and eating all the good biscuits.
I wonder if this is accurate as to geography. Wikipedia says "The current boundaries of Hungary are the same as those defined by the Treaty of Trianon [in 1920], with some minor modifications until 1924 regarding the Hungarian-Austrian border and the notable exception of three villages that were transferred to Czechoslovakia in 1947." Also "through the 1947 Paris Peace Treaties, Hungary was again reduced to its immediate post-Trianon borders." On those statements, any territory that is in Hungary now has been in Hungary since 1920? Are we looking before 1920? Thanks in advance. ;-)
"as part of the "new employee induction" had to go on a training course to learn how to use an application I'd designed and coded."
I've had that happen twice, except with hardware. One of the jobs I received mainly due to the fact that I worked on the engineering team that designed a number of machines the company used. It would have been fun to purposely fail the class to see what would happen.
You can’t be someone I’ve assigned mandatory security training to, then.
The provider we selected (SafeStack Academy) let you skip directly over the lessons — the entire ridiculously long and cumbersome 5 minutes worth — right to the quiz, which if you pass, marks the lesson as complete.
There are some who complain. “Why should I take mandatory training when I know this?” Well skip to the quiz and answer it in 30 seconds, as I did. We can never assume competence.
"I had to organise a course on how to fit a plug on a mains lead for my comms team"
Many years ago I had to take a first aid course to get my driving license. The guy running the course was a professional paramedic with years of training - but had never in his life taken a simple four hour "first aid" course. So when he wanted his driving license, he checked and he wasn't allowed to take his own first aid course because he couldn't be the course trainer and a course participant at the same time; he had to take a course run by a colleague.
Hah. I own a house which was professionally rewired and certified for letting before I bought it. When I had it re-checked the new sparky found that two double 13A outlets had been fed, via 0.75mm flex, from the lighting circuit.
Installed by a professional, checked by a professional, signed off by a professional.
Argh! Part P was such a pain in the early days.
Having a degree in Electrical Engineering, and subsequently becoming a Chartered Engineer, I've rewired most of my houses over the years. That all came to a stop in 2005* when I was forced to employ a sparkie to replace my consumer unit, except that the 'young lad' couldn't figure out how to get my external lighting working.
"Move aside if you will" and a quick rummage later everything was back on - the PFY had never experienced the strange type of wire used by the MAFF** back in the 1930's. I told him to just sign the paperwork and go away.
Fast forward to today, and my current house still has wired fuses and an ELCB because I have to pay for a 'professional' to change it all. It's on the todo list, before I get that EV on the drive.
* Part P came in to effect in 2005, and I also became a Fellow of the IET, so felt duty bound to abide by the rules
** Ministry of Agriculture, Food and Fisheries - it was an ex-government building.
I have a degree in Electronic Engineering (didn't take the right module in the final year to get "Electrical and Electronic") and have always done all my own wiring. I changed career to be a self-employed domestic electrician a couple of years before Part P and although I did take "The Regs" (C&G 2381) before starting, when registration came along the inspector took my qualifications at face value, looked at a couple of jobs I'd done (including the house I was then living in) and signed me off. I had to have test equipment, but they didn't even require the C&G in Inspection & Testing (2391?)
Years passed and I got a salaried job and let the Part P slide. We started a project to build our own house and I was always going to do the electrics (and the plumbing, but that's another story) and looked into re-registering. The list of mandated training courses is now as long as your web browser window (and I can't count my degree any more) and with the registration fee would have cost almost as much as getting the job done by someone already in the business, let alone the timescale to study the courses. Oh, and the mandatory requirement to have some work to inspect means I have no clue how someone can start up on their own unless they can find an existing electrician to work with first!
Easy, I thought, I'll do the grunt work myself and get an electrician in to inspect as I go along and sign it off at the end.
Very, very few electricians are willing to do this because signing it off means putting their professional name to the job and thus their own insurance, and makes them liable to come back to fix things. One electrician wanted to be on site with me every day I worked, which wasn't viable as I was doing this mainly on weekends and evenings.
Eventually I found someone who was willing to come a couple of times - first to approve my plans, then to check (before boarding and plastering) that I hadn't run cables in stupid places, then (once wired up) to do a whole-day thorough inspection & test, which involved dismantling every fitting.
And because he was up-to-date (I had last been registered ten years previously) he pointed out a couple of things I'd missed, even though I had read the latest regulations. In particular the requirement to use "fireproof" fixings where cables run across egress routes - so metal cable clips, or metal bands around bundles, that sort of thing.
And still I look around and think, "if only I'd fitted a double socket there instead of a fused connection unit"!
M.
EEs can perform house wiring in a very acceptible manner. They can also introduce major hazards because they "know electricity" but don't know code.
For example: an electrician might not know why the grounded and grounding conductors shall only be bonded at the service panel or at a separately derived system and shall be separated at a main panel, but they will follow that rule. An EE is likely to think "well, ground and neutral are connected together at this panel, so it's just a good or even better to connect them at that other panel as well.
Source: I have a BECE and have also successfully wired a new house, including passing the electrical inspection.
"Easy, I thought, I'll do the grunt work myself and get an electrician in to inspect as I go along and sign it off at the end."
A big portion of a job is often drilling holes and running cable. If you can do that properly, it can be a big savings in labor. Just leave the important connecting up to the sparkie so they're not too far out on a limb. I had a low voltage license to do home theatre and alarm systems. I found the courses to be really boring and far off of the mark from the real world. A big problem is regs don't keep up with the state of the art.
The house we just bought had two subpanels, one with breakers and blank plates glued in place with Gorilla Glue.
It also had one ceiling light that didn't work. The electrician replacing the panels traced the fault to a wire that had burned through- it had been stapled so tightly to a joist that the wires shorted out and burned through. Said electrician came down from the attic jubilant with said cable - he now had another example of what can cause fires for his friend, the Fire Marshall, to use in his safety talks.
Its always management coming out with 'The Unions Wont be Happy'. The time I spent active in a union we saved the company more money by knowing the company and country rules than any of the management who came out with this crap ever managed let alone made for the company.
... was point out to his Boss, that he could perform an internal charge at Callout rates (plus plus plus...) to the Infrastructure Team for fixing their Failure. If the Infrastructure Team objected, pointing out to the CEO that had the failure not been corrected when it was, the loss of production would have cost $X (almost certainly multiple factors of 10 higher than the Charge on the Infrastructure Team). Of course the Boss could also then request additional resources for his department as reward for keeping the wheels spinning when other teams where letting the company down...
Solidarity in management only extends to the point where one's own department could get a leg up...
What Derek needed to do...was stand up to his boss and tell him, in no uncertain terms, that he was not going to listen to a dressing down as if he was a small child.
"Boss," Derek should have said, "what I did may have been outside the lines, but the plant was down and in my professional judgement, I did what I felt needed to be done to remedy the fault. We may disagree about that, but let's discuss it like adults. If you can't do that right now, please call me when you feel you can"
Obviously not in the same league, but I once spent a couple of hours trying to get my laptop plugged into the big meeting room screen. But my HDMI cable seemed dodgy (which was odd, because I'd used it elsewhere without issue). Tried another, and another (by this point I'm "borrowing" them from vacant screens in the office outside). Some just don't work, others drop the connection every few seconds.
Eventually, I realised that the HDMI cable between my laptop and the wall socket was not the only one, and checked behind the screen to find one of the plugs of the other cable wasn't fully seated.
Then I just had to return all the borrowed cables...
When she was younger, one of our cats took a liking to curling up behind my PC tower to take advantage of the nice warm air venting out of the PSU, but in doing so she'd always end up getting partly tangled in the loose cables hanging down the back of the tower from all the stuff plugged in above the PSU - USB, audio, video capture, and DVI...
One day I discovered, to my utter delight, that the repeated tugging and pulling on one of the DVI cables had caused the corresponding socket on the graphics card to fail (I'm guessing it popped a solder joint, just as happens from time to time on headphone sockets after a while). Fortunately this was on a card with more DVI outputs than I had monitors, so the fix was to a) move the cable onto one of the unused outputs, and b) invest in some cat-resistant cable management hardware (aka heavy duty cable trunking large enough to swallow the entire spaghetti factory of cabling required by the PC) to minimise the amount of slack cabling left within reach of curious kitten paws.
Around the turn of the millennium I did a contract stint for a local authority on a project to hook up all their schools to the internet. The job was split into phases – the comms crew would hit site to install the CPE, a week or so later the cabling guys would arrive to CAT5 the building and then my team would turn up to install the (singular) Cisco switch and get some lights blinking. Another crew would follow in our wake to install the (singular) school server and back office PCs.
On one occasion me and my (staff) partner arrived at one site to find that the cabling crew hadn’t plugged network cables into the patch panel in the comms rack, where we were installing the switch. Instead, they’d just left a box of unopened cables in the corner of the room. My sidekick immediately pulled out his phone to talk to his manager about “the problem”. I literally could not believe what I was hearing. I pointed to the box of cables and mimed opening a zip-lock bag, uncoiling the cable and plugging it into the patch panel and switch. Still in deep conversation with the boss, he ignored me. So I went over to the box, picked out a cable and pulled the bag open. At this point he bellowed STOP!!!! at the top of his lungs, muttered something into his phone and thrust it into may hand. At which point I got told in no uncertain terms that I was not, under any circumstances, to plug in the patch cable, as it “was not our job” and that the cabling crew would be scheduled to go back and plug the missing cords into the patch panel.
We then had to explain the problem to the school secretary and apologise that it would be another 4 – 6 weeks before their school would be connected.
I vowed to never take another public sector contract again.
Doing the same job for a county council at the same time we would send out a couple of junior techs with a van full of equipment and cables.
We would connect the comms gear up and soak test both ends of the connection in the office a couple of weeks before so all the techs had to do was plug it in and cable everything up.
It was a system that worked very well and cost effectively until one of the techs drove a transit full of equipment into a dyke during icy weather. Unfortunately we had opted to transport the kit unboxed as we needed to get 30-40 PC's in for each secondary school at the time. we'd also make sure we had cabling contract cover on standby and our comms team available over the phone to sort out any teething problems on the day.
At the other extreme I was doing some comms rooms audits for a large government department. They had been ripped off by their service provider and I would actually find piles of un-racked powered off switches they had paid to be installed by the outsourcing company. In one call center the network wend down while I was there. The on-ite support tech employed by the department was not even allowed to go into the comms room now support was outsourced. I found a frayed cable running out of the front of the comms cabinet which had the glass cabinet door resting on it. I could see the cable ran into the fibre switch on the other side of the room so found a patch panel and replaced it. All hell kicked off from the service provider when the site appeared on the network without their intervention. As I worked for a central trouble shooting team and had already reported what i had done to the Assistant Director who ran the site this was rapidly put down but the on-site techie would have faced disciplinary action. I did report back the the whole room needed recabling to a decent standard and running the single connection tot he fibre switch outside of the false floor was completely unacceptable bt that didnt take place during my tenure there.
I vowed to never take another public sector contract again.
In defence of the public sector , thats not typical.
In fact i would imagine you'd run across that kind of bloody mindedness more often in the private sector , especially when dealing with contracts between compaines
In fact i would imagine you'd run across that kind of bloody mindedness more often in the private sector
That's the complete opposite of my experience, but then I've never worked for large corporates(*), only small companies where the attitude is "if you can fix it, do so".
(*) To be precise, I worked for one of the biggest(**) for ~3 months after a buy out. My resignation went in about the time the US corporate management told the local UK management to introduce the company song book.
(**) So big our nine digit acquisition price made a 1 ULP difference to their quarterly report figures.
Ah yes, public sector unions.
We got a nice new minicomputer for the engineers to use, saved us having to use 300bps dialup modems to a big IBM somewhere. Lots of VT100 terminals, very nice, until the unions discovered that we wanted them on our desks. That was not on, only typists had keyboards on their desks, not engineers. We were only permitted to use terminals that were installed in a designated terminal room (with the modems). Seems they didn't count as keyboards, wtf? We were threatened with dire retribution by the union members if we dared break the rules.
Screw them, 90% of us weren't in the union anyway. We put the VT100s on our desks, pulled in the cables through the ducts ourselves. One or two union loyalists refused to use them, but after being left sitting alone and laughed at in the terminal room most gave up.
We fetched in a contractor about that time to help us get all of our schools on - we had our own cabling team, and two subbies, but if something like that happened round here you’d have been told to crack on and get it sorted - our bosses were reasonably pragmatic about it. I think the only install that didn’t get completed was one where the patch panel hadn’t been fitted for some reason!
Strangely, I've never been able to identify the boundaries of the SEP field and obviously I'd never knowingly circumvent the system ...
I always exist in the IBFI (Its Broken Fix It) aether ... "You're a computer tech - fix my computer chair ...", "You're the technician, fix my glasses", or even "You're the technician you can fix my intimate hair trimmer"
Too true!
I was once sent to buy a kettle because the downstairs wall mounted hot water heater had packed up.
IT weren't the ones who noticed nor reported this - just a decree from upper manglement to the head of IT to get one of the team to Makro to buy one from petty cash since they didn't want people walking up and downstairs to make a brew.
Unfortunately for me, my missus is one of those people who bug me with:
1) You're an engineer. the Internet isn't working. Why can't you fix it?
2) The smart TV isn't working. You're an engineer, why can't you fix it?
3) The washing machine isn't working. You're an engineer, why can't you fix it?
etc etc etc.
It wasn't, by any chance, made by an Italian manufacturer, the name of which starts with a 'Z'?
I've heard those have reliability problems.
With white goods, either go uber-cheap and expect to replace every 3-5 years, or buy a more expensive German-made one. (e.g. Bosch)
Forget the very expensive designer ones, with mildy amusing names, they invariably aren't worth the extra money for the sake of looking pretty and having a big advertising logo in the middle of your kitchen.
It was a Samsung, not a cheap one - we have a couple of holiday cottages and needed a serious bit of kit to cope with vast amounts of laundry some people manage to produce! The other half is seriously diligent when researching these things and it should have been a good buy. Even managed to get some engineers round to fix it under guarantee who seemed to know what they were doing and it still fought back. One did try to recommend a whole new replacement but got overruled. We know have a new expensive one which so hasn't thrown an error yet but I fully expect it to be a well out of band error when it does, Things either work as expected here or just fail in ways that baffles the fuck out of everyone. I do wonder if its being in the country at the end of a lot of wibbly wobbly mains feed, regular lightning outages and frequent low voltages.
I suspect that some families produce more washing than others. For example, those with small children, staying at a holiday cottage for a week. They may not want to bring multiple suitcases full of clean clothes with them for the Little Darlings when they want to play in the mud.
Today's (17/01/22) ITV4 repeat of British spy-type show "The Avengers", also tomorrow with sign language at 08:05, is "Stay Tuned", in which spy guy John Steed plans a holiday but is grabbed by brainwashers and finds himself three weeks later still thinking he's just setting off. Tara King, setting him right, asks whether he would take a suitcase of dirty laundry on holiday, which is what he has. So the answer in that case is no.
Hotpoint seem to do a good job of self combustion. I got a call from the Mrs one day to say the washer was giving an error. A quick Google suggested it was the locking switch, so top came off to reveal lots of very crispy wire and a blackened smudge where the lock switch should have been. At that point, I downed tools and rang Hotpoint. To their credit, an engineer was soon dispatched to us who asked if we were happy to have the switch replaced or if we wanted a new machine! Thinking a new machine seemed excessive, we let him replace the switch, although the old ones remains went in a padded bag to be posted back… about three weeks later, there was a general recall campaign and all of those washers were “not to be used, and we will replace your machine”. However, due to the sheer number of replacements needed, we’d be without a washer for umpteen months, so we decided to get a new one elsewhere and sell the replacement from Hotpoint when it arrived.
While trying to find a replacement, they recalled the tumble dryer as well, so I effectively bought myself a washer and dryer for Christmas that year.
This post has been deleted by its author
"Seals are surely the purview of marine biologists."
A friend of mine used to run the in-house translation department (when there were such things) of a large oil company. One day he got a phone call from an angry freelance translator complaining about them clubbing seals to death to use their oil in the refinery.
Turns out the document to be translated contained a reference to "seal oil", which is a barrier liquid pumped through sealing components of rotating equipment. Guess the translator had a language degree rather than an engineering degree.
Here's one to start the weekend -->
Seals are surely the purview of marine biologists.
It's Friday, so this is the only possible reply.
Reminds me of a time when I literally got my mate to fall off his chair he was laughing so hard:
We were talking about seals (in this case for air rifles and I mentionned that the seals came in a pack
'What's a seal pack?' he asks (meaning the price)
'About 2 1/2inches' I replied.
I fixed a lady-friend's washing machine, it was flooding and her husband's plan was to buy a new one. Husband was away on manoeuvres with the territorials and we were taking advantage of his absence. The bed-sheets needed washing so I took the machine to bits. A coin was blocking one of the drainage pipes. My mistake was to leave the retrieved coin on top of the machine, hubby noticed it on his return. His pleasure at finding the machine had been serviced for free didn't do much to reduce his annoyance with me for "servicing" his wife (also for free, seemed a fair trade to me...)
Hell, I just ripped apart my electric stove/oven combo, ordered the new part and swapped it in. The $400 I spent beat the $1200+ it would have cost to buy a new unit and have it ordered, plus the the time I would have spent looking or a new unit, negotiating with SWMBO over features and looks and such.
And I'm still cursing the cheapskate who didnt' put shutoff valves around the FHW (Forced Hot Water) heating system zone pumps, so they could be swapped out when they die without draining the entire damn system.
And when I got a guy in to quote me a new heating system because they noticed my oil tank is leaking a tiny tiny bit (in all fairness, the system is probably 40+ years old) he was surprised I wanted those types of valves, because most people were too cheap to "do it right" for his company.
The other guys said they "only" do it right from the get go, since they have to maintain it.
Yup. A company I worked for had developed a digital printing system for the foil that goes on the back of tablet/pill blister packs. It had stopped working after the pharmaceutical company it had been supplied to had moved the packaging machine it was attached to and they needed to get it working again. All the people who had worked at our end had moved departments so I - the relatively new guy - got sent to the US with a laptop, the source code and the old compiler that was needed.
Took me a while to hunt it down and, in the end, I had to resort to tracing every single wire from every single sensor and, yes, someone unknown had removed a panel which had a ribbon cable attached. Obviously the ribbon cable was too short, they'd half yanked it out of the IDC clip and then quietly put the panel back. So I carefully removed it, quietly took it to a vice, gave it a nice squeeze and then put it back. Everything now worked.
Not sure what the bill was (consulting rates, flights, hotels, car hire, etc.) but that was probably pretty expensive for just using a vice.
Or to be more accurate "for knowing which bit to squeeze in the vice"...
I never forgave them for turning Trillion from the cool, intelligent astrophysicist of the radio series into the lisping airhead bimbo of the TV version. Zaphod's second head and third arm were also rubbish, but it was the casual sexism which most annoyed me.
I think the HHG book series was at least one book too long and Douglas Adams clearly thought so too; my guess is that's why the 5th book ended by killing everybody and destroying the galaxy just to make sure that nobody could make him write a sixth volume.
But, IMO the best HHG version was live onstage at the Hammersmith Odeon: Vogons leaping off the stage to beat up the audience, cardboard police blasters shooting real laser beams during the shootout that blew up the computer-bank, causing everybody to end up in Milliways. Yes, it was even better than the radio version. Its only downside was that by the time I saw it the bar had stopped selling Pan-Galactic Gargle-Blasters in the interval.
Zaphod was played by two actors in the same clothes, so his heads could and did argue: much better than the Beeb's patheric, obviously dead plastic second head.
Didn't quite have the same experience of an adaptation to a stage show at The Drum in the Theatre Royal Plymouth, don't recall how they dealt with the head issue, the audience became the frozen Golgafrinchans at one point.
The thing that really pissed me off was as it was the matinee performance, the production skipped over the Milliways scenes (I still have the program souvenir (Not the t-shirt) that listed Garkbit, the Dish Of The Day etc).
"my guess is that's why the 5th book ended by killing everybody and destroying the galaxy just to make sure that nobody could make him write a sixth volume."
But the 6th volume was written, only not by Mr Adams, instead by Eion Colfer although the style is pretty similar.
The vinyl LPs were (probably still are, if I knew where I'd put them) Jolly Good Fun as well. Nicely demonstrated how Mr Adams was unable to restrain himself from never quite telling the story the same way twice.
Mine has the yellow duck in the pocket.
Part of the variation of course is the bits of the radio one that John Lloyd wrote, and that I gather Douglas Adams had second thoughts about having a co-writer for book publication.
It's also one of "those" programmes where we hear that the last page of script was being typed while the actors were performing the start of the scene. And they only had the studio till go home time.
This post has been deleted by its author
I remember listening to the radio series as a kid (my parents were fans - or at least my dad was). I read the books many times and watched the old TV show (which was "entertaining theatre", but nothing special).
I managed exactly three minutes and 42 seconds of the movie before switching it off.
Whenever I realize that I've just said "It can't be this bad!" for the eleven-dozenth time that day, I resort to H2G2. After a few pages and a smile, I sigh "I seem to be having tremendous difficulty with my profession" and hope a freak wormhole opens up I can jump into.
Not to the same standard as some of the other comments, but as a humble sys admin (who is part dinosaur/part furniture as I've been around a while) it seems I frequently have to point out to our devs how their applications work when they screw something up in the config. Usually it's copying something from one env to another without actually knowing what all the config does. Sometimes it's complaining that something suddenly doesn't work and it must be a system problem, even though it was working fine then they changed some config/jar or whatever and now it doesn't work. Or asking me why something doesn't work without doing something useful like looking in the application logs where there are obvious errors.
All SEP as in most cases I'm just responsible that there is a system, that Java is available and that there are JVMs for them to put stuff onto. As to what the JVMs actually do, well that's not my sphere of influence...
One day, I came into work to find we had no network in part of the building. It was a part of the building without many people in it, so we didn't get any complaints, and I only noticed a problem because I tried to log into a machine in that part. I went into the comms room, and found that the switch connecting that part to the network had hung. Quick as a flash, I thought I'll power it off and on again. So, I did. When it was plugged in again, the switch was still hung. So, I tried again. Still no difference. As it turns out, I had pulled the wrong plug.
In my defence, these switches had no mains switch on them, and the cabinet was tightly packed, so it was difficult to reach around the back for the mains connector. I could reach the power strip at the back of the rack though, and traced the power cable to what I thought was the right socket. A difficult job due to a combination of the fact that every mains cable was black cable with an IEC plug on one end, and a standard 3 pin mains plug on the other, and the fact that there was almost no light in the back of the cabinet. There were several identical switches in this cabinet.
Anyway, I digress. I had powered down the wrong switch twice now, so I powered it up, and left it to boot. As anyone who has dealt with them will tell you, Cisco enterprise switches do not boot quickly, and it seems even longer when you are anxiously awaiting the boot up.
I had good reason to be anxious. Instead of a switch that was lightly used, and not actually working, I'd accidentally powered down a working switch that was very heavily used (every port patched, with a light on). Thankfully, no one complained. Not even our networks guys, who have a system monitoring all the network infrastructure so were getting warnings.
I eventually did find the right switch, and rebooting it did cure the problem, so all was well.
A difficult job due to a combination of the fact that every mains cable was black cable with an IEC plug on one end, and a standard 3 pin mains plug on the other, and the fact that there was almost no light in the back of the cabinet.
That's where having a tiny zip tie or somethingequivalent is really useful. Wrap it around the cord at known end and slide it down to the other end. Of course if there are any tight bundles that its unlikely to work very well.
Dell PowerEge servers (love 'em or hate 'em) have a pair of buttons on the front and back panels that cause a pair of lights to blink, front and back, so you can push the button in the front and after walking to the back of the rack you can find which server you're actually supposed to be working on.
Early finish Friday afternoon, last people to leave, and the burglar alarm can't be set: 'Fault'. We'd had some problems with intrusion attempts and with a deep sigh we called the 24-hour service company. Being the hero, or drawing the short straw, I stayed behind to await their arrival. To be fair, it was only an hour or so wait and the who guy turned up was also anxious to get the problem sorted and get on with his own end-of-week refreshments.
He quickly diagnosed a cable fault, which drew a groan as the cabling had been installed by an artisan better suited to serving spaghetti. Nevertheless, the fault was deemed to be 'between buildings' and he prepared to pull a new cable through; this was going to be a long job. I suggested that we inspect the access duct-covers first...... Well, the fault was obvious, something had chewed through the cable. The burglar-alarm detective said "That's a *very* big rat." He also advised he was not going to put his hands in any duct containing "that"..... We, meaning I, had to make some other sort of provision. His problem became mine.
I retrieved a pair of welding gauntlets and pulled sufficient free spaghetti/cable up to the surface for repairs to be effected. And a large board to place over the hole temporarily. The repairs worked, the alarm was set and the weekend started a bit later than usual.
The following Monday, I told the factory guys about the event and they said they had seen this rat and confirmed it was 'bigger than you'd like to meet'. Cue call to a well-known exterminator.
Rats do occasionally grow to extreme sizes. I bought a place with a badly designed chicken house - 6" off the ground so ideal for them to be able to hide underneath feeding off grain that had time to chew through wood to make holes in the house and somehow encourage the chickens to put their heads through and eat them off. A cat from the next door farm adopted us and it was a little cutie but a ferocious ratter and the remains of rats we found about the place grew as it became more experienced. It eventually caught the mother which had haunches a rabbit would have been proud of and an 8" tail that was a good 3/4 of an inch thick where it joined the body. the upper body and head were missing but I think it was bigger than the cat before she and the jackdaws devoured most of the remains.
Coming through Sidcup station (An area packed with fast food outlets, restaurant's & drama students) with a fellow commuter as we approached the sensor operated doors, the biggest fucking rat I have never wanted to see ran past the doors on the outside.
(Icon) Did you see the size of that thing?
Yes, was it a rat?
I think so, it was about the size of a smallish dog*
Want to to give it a minute before we go out?
Yes!
*I don't mean a Yorkie or similar breed either.
I retrieved a pair of welding gauntlets and pulled sufficient free spaghetti/cable up to the surface for repairs to be effected. And a large board to place over the hole temporarily.
Out of curiosity - how long did the "temporary repair" last?
If it's like mine - probably still there!
"That's not your job!" snapped the boss.
Christ, if I had a quid for every time that scenario came up when I was in the rat race, I'd be retired now, too.
It used to manifest itself in many ways. My favourite was when you changed roles, but your replacement in your old department would steadfastly refuse to ask for advice if he or she encountered problems, and said department would end up having to reinvent the wheel to fix things. It happened time and again.
A massive waste of resources.
Well, not seriously, anyway.
On the other hand, I have occasionally been instructed to NOT fix something, since the root cause lay elsewhere, and the team/person who owned the system in question needed to get out of their well-padded chair and deal with it.
In fact, I'm doing something similar right now; a customer was informed a few months ago that something they were doing wasn't supported, and that they needed to switch over to the recommended process. Which they duly promised to do. In writing.
Fast forward to now, and when I got a ticket saying "the unsupported thing isn't working", there may have been a little schadenfreude when sending a message back going "Sorry, but as per <copy of email chain>, we ain't touching this as you need to do what was agreed on several months ago"...
I've been told off for doing work that was...better than desired
Most recently, we had to configure a lot of desktop PCs. The schedule was 5 each day, for several weeks. Life is short. I set up 14 at once and went back to my desk, RDP in occasionally while I do my regular job. 3 days of installs each afternoon.
The project manager got upset at me for screwing up his schedule. I told him to piss at my boss, because I've never been written up for delivering early. Thus ended the matter.
I've got one currently with Microsoft. We're using Azure CDN (to cache stuff) and it's throwin an error. They've done some diagnosing, and it turns out the error goes away if you disable caching (so it still queries the source for every request). When I asked what the next step was they said "I'm glad to hear it works, you can leave it configured like that"!
Technically they've fixed the problem, because there's no longer an error...
another true sounding story...
Its when you look at $contracter's code and think "well thats a pile of dog do do" and you change it so it actually works (hint: PLA logic does like pauses between gate shut commands and gate lock commands for the freaking gate to physically shut ... sheesh the guy was a 'pro'
then you alter it and get dragged off to the mangler's office to be yelled at that you cant change $contracter's code because its not our job.
"fine" I said "I'll put it back as it was and the line can stay stopped until you get $contractor to fix the shoddy code"
Many, many years ago, I had a job where I was nominally expected to supervise a more junior member of staff. How to log in, which system did what, how to edit a file, basic shit like that. Anyhow, this being the days of mainframe and COBOL, the idea was that they would learn on the job, eventually becoming familiar with the basics and moving on to understand the basics of the various systems (many running in batch) that held things together.
Part of the job would eventually to be expected to be part of the rota that dealt with outages encountered out of hours. For this, your basic equipment was a series of cabinets, running the length of a corridor, containing the latest compile listings of the various programs in the suite printed out on music rule paper, each snug in it's own individual binder.
Batch Mainframe COBOL being what it is usually used for (financial stuff), the most common severe problem encountered would be a program abending with a S0C7 abend code (*usually* decimal data exception), due to badly sanitised input data masquerading as a valid decimal value, the failing instruction commonly being reported as ZAP (zero, add packed), the usual way to initialise a decimal value with something coming in from whatever source the program used.
Of course, this level of WTF is not something that a trainee expects since all of a sudden one is thrown in at the deep end of the somewhat huge disconnect 'twixt a high level language and the underlying machine architecture of the brute and bears little resemblance to whatever they might have learned beforehand.
Anyhows, one of the other trainees found themselves completely baffled as to how to diagnose such things and mentioned this to me. Since when I was in the same boat some kind person had explained this stuff some years before, I sat them down with a compile listing, explained how the compiler translates COBOL to machine language, what the instructions mean, and how to relate this back to which COBOL instruction and which input field was the culprit (and how to find that).
They got the idea and were happy and cheered up, now being more confident that they understood something useful.
Shortly afterwards, I got a visit from our "training manager". "Why did you explain that to someone else's trainee?".
Well why not, I ask people how things work all the time, and they explain them, and you don't tell them off (they were all well above her pay grade).
"No, that's not your job".
Said company is long gone (wonder why) and by a strange co-incidence, I've just fixed a S0C7 abend at a customer site (a bank, no less). It wasn't a decimal data exception, BTW (bonus points for anyone replying who can identify what it was) and everyone was happy, but to this day, I cannot understand the mentality of criticising looking after people or acting for the greater good. I've found, over the years, that getting away from that shit rapidly is a smart career move.
Life's too short to work for such a person.
Been there, done that, got the hell out. Now I work for people who reward, rather than punish, doing the Right Things.
A place I once worked was acquired by GE; we did little 110V and battery-powered instruments. And were ALL required to take electrical safety courses wherein we learned such highly relevant skills as how to use a hot stick on a 15KV utility line, and which color gloves to wear for what HV lines. In case we had to maintain a 15KV line for the portable instruments, y'know.
Oh, and they also required us to undergo Mandatory Integrity Training. Whereupon I filed an integrity complaint against Jack Welch and his just-revealed secret golden handshake. Eventually resulting in amusingly Kafksesque integrity theater as they unconvincingly pretended to follow their own required procedures while not actually living up to their professed standards.
Posting anonymously because, well. Those who were there will know who, the rest don't need to.
When I joined BT as a software engineer in the early 80s we were all sent on the standard engineering training. We learned to climb poles, do gas safety checks on manholes before lifting the cover, and handle high-voltage cables. Essential knowledge for Fortran programming on an IBM, I'm sure. I still have my blue hard hat.
Some years ago, I got loosely involved with a local amdram group. I’d been playing in a band for one of their shows and the sound guy was awful, so after the run, we were talking and I agreed to run sound for their Christmas panto.
About two weeks before the panto, their lighting guy (tech in a different theatre) announced he couldn’t do their show, so the amdram chairman decided I should do it as “it’s all just sliders and wires, isn’t it?”
I quickly figured out the static fixtures, but the moving lights were a bit trickier, and took most of the weekend to get programmed into the lighting desk at all, never mind actually working as something theatrical… however work they did, and the theatre manager, duly impressed as they’d never worked before offered me some part time work as a theatre tech. Well, that seemed like fun, so I joined the payroll, but that meant I had to do the induction training of the group. Which had a single theatre, but around 25 swimming pools and leisure centres, so lots of info about what to do with chlorine, and how to deal with an escape of chlorine, nothing at all about ladders or electric…
After a few years, I suggested that two of my colleagues went on a theatre tech course. H&S objected - they’d be taught about electricity, which was too dangerous - was there not a better way to light the theatre?
The local networking IT was outsourced overseas. An entire office floor was off-line. The remaining desktop IT spent an hour talking to the overseas network IT who had access to the switch gear. I was tagging along because, well, with the network down I had nothing else to do.
In the comms room I looked at the (then new tech) orange fiber optic cable coming in from the other floors. I innocently asked "Shouldn't there be a link light". Overseas network IT got quiet but I could hear key clicks on the speaker phone. A moment later the link light popped on and everything magically started working.
Most of the staff in this sales office got a half day off, which averaged several million is potentially lost revenue. But hey, we saved a few bucks by firing the local IT network staff and outsourcing. So it was a good thing.
Does insulation really matter when they can chew through almost anything? Or is the insulation something they find unpalatable, but they somehow find the raw glass/plastic fiber tasty?
A while back I had issues with mice getting into my house in the basement ceiling (fully finished, with drywall in most of it and drop panels in the rest) and while I was fighting that battle my DSL was having issues. Turned out there was a "splice" in the telephone wire in that drop ceiling I wasn't aware of that was made with electrical tape, and apparently a mouse had chewed through that. Basically my entire DSL was operating over what appeared to be a single solitary strand of the stranded copper wire. When I cut off the ends and used those 3M snap on crimp things to connect everything was fixed.
While the mice are gone for good (so long as I found the ONLY way they could get in) I can't help but worry what I would do if they chewed through the wire insulation somewhere above the drywall ceiling. I wouldn't even be able to figure out where the break was, let alone fix it without cutting several (at least) holes in the ceiling to fish through a new wire! I should probably just move to be safe!
"I can't help but worry what I would do if they chewed through the wire insulation somewhere above the drywall ceiling. I wouldn't even be able to figure out where the break was"
That would be the perfect excuse to treat yourself to a nice Time Domain Reflectometer. Used to cost more than a car, now cost less than a bicycle (pedal, not motor).
Here's an example: https://nl.rs-online.com/web/p/time-domain-reflectometers/2261537
Explaining that it's not a cool toy but a tool essential to fix the intarwebs normally greatly enhances the spouse acceptance factor (if that's relevant). (Hmm, beginning to regret mouseproofing our basement now.)
I just had Openrech round to find and fix a break in my phone line.
They used a TDR but were over 50 feet out, they went up the pole outside, into the manhole (needed a 2-man team with breathing apparatus), to the exchange, and back again several times, on multiple visits, before they eventually found it half way up the pole.
When I worked for a local government we, as in the users, designed a database app based on what we actually did. Since none of us knew how to use any of mictosoft's code products, we used a spreadsheet with macros as our demonstrator. We tested it with actual data that we compared to what we were were using. Pretty as a picture, very adaptable and had a lovely GUI. Our in house IT staff thought highly of it as well.
Unfortunately, all we got was an adulterated version of another database we used in a different department that was universally hated by them.
Here's the kicker. A couple of us would regularly find key combinations and shortcut the provider didn't even know existed. And several were quickly erased. I guess it doesn't pay to show up the "experts" or point out the errors and fallacies their in-house "pros" created.
Only vaguely related, but I once found myself as an occasional sub sub sub contractor to a chain of high street shops that oddly no longer exists.
Shop floor staff were under no circumstances allowed to touch IT equipment. This kind of makes sense.
The unintended consequence was that when a broken USB barcode scanner needed swapping for a new one, I had to drive, on one occasion, 90+ minutes each way. By the time my invoice for several hours for a two minute job (20 seconds to swap, then wait for someone to test it) had been passed up the chain to head office it probably wiped out the shop's profit for the week.
My boss noticed me doing a bit of soldering on an electronics kit during my lunch break. "You obviously know something about electricity could you have a look to see why the lights in the ladies' toilets are always blowing their circuit breaker?". I quick check in the false ceiling showed a broken cable shorting out on the metal fixings for the ceiling. Snip snip with some wire cutters and a £2 junction box from a local shop I had the problem sorted.
A million years ago (well, the 60's...) a friend of mine was working in an 'electrical shop', where back in those days sold appliance which often didn't have mains plugs on the the power lead as there were still a lot of houses with old two pin sockets. When you made your purchase you told the shop assistant what plug you required, took it home and fitted it yourself.
One day an elderly gentleman comes in to buy an electrical kettle. My friend says fine, what plug do you need. The gentleman says three pin please. My friend tells the chap to very careful wiring it up and make sure it's earthed properly. He came back a few days later complaining the kettle didn't work. My friend opened the plug to check the fuse and discovered he's put all three wires onto the earth pin....
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2023
