back to article You want a reboot? I'll give you a reboot! Happy now?

Today's tale from The Register's Who, Me? files is a reminder that a momentary loss of focus is all that is required to trigger a potentially catastrophic error. Our contributor, Regomised as "Sam", regaled us with a story from a mere five years ago when he was still a fresh-faced worker doing time as second-line support at …

  1. Joe W Silver badge

    Background

    And that's why some of our more important systems had a garish pink desktop background when logged in as admin.

    And the Linux server had a specially crafted command prompt, nasty colours and all.

    Yes. There is a reason for this. Yes same as in the article. Yes, the boss was not pleased but understanding (just don't do this again, were his words).

    1. Ben Tasker

      Re: Background

      I sent a corporal off to the server room to restart one of the boxes (it was taken out into the field on exercises but otherwise just sat their and maintained).

      On his way there, a WO2 pulled him aside and bent his ear about some stuff that the WO2 felt was outstanding.

      So, said corporal went into the server room and went onto the production system (which was supporting active operations), checked the tasks the WO2 had insisted on, and then issued the reboot command...

      When shit hit the fan, my position was that it was an easy mistake to make, and that the jobs the WO2 had tasked him with

      a) weren't actually outstanding

      b) weren't that important

      I also pointed out that, as a civvy, I'd have told the WO2 his tasks needed to wait, but a lowly corporal doesn't really get that luxury so the WO should perhaps have spoken to me instead.

      Corporal got away with a bit of egg on his face, but no other real consequences.

      1. Anonymous Coward
        Anonymous Coward

        Assumed Rank

        I did some work for a police force years ago, their policy was all consultants were given an assumed rank which was higher than everyone working below them on the project. As I was providing operations management consultancy that mean I had to have the same rank as the Chief Superintendent who managed the DC. This apparently gave me the right to take the tech team into the silver service senior officers mess ( I was actually taken to one side and discretely begged not to do that).

        The end result was that if anyone wanted to do anything which could potentially affect my work they had to consult me first and get my explicit permission and no sergeant or civilian could be ordered to do something without my say so.

        1. ARGO

          Re: Assumed Rank

          I had the same when doing stuff for the military. I felt rather awkward that the rank they lent me was three grades above the level my grandad reached after 5 years of active service.

        2. Frank Bitterlich

          Re: Assumed Rank

          Wow, that's the first time I heard that rules were put in place to actually *consult* the consultant. Seriously, IMHO the standing order more often seems to be "hire a couple of expensive consultants and then completely ignore what they say."

          1. John Brown (no body) Silver badge

            Re: Assumed Rank

            That probably depends on if the consultant has been hired to do an actual job or just to produce a box-ticking report.

      2. Eclectic Man Silver badge

        Re: Background

        I was on the receiving end of a mis-typed command once. There I was happily playing, sorry doing my work, on my Sun Microsystems workstation (high resolution monochrome screen, loads of RAM etc.) when it just literally stopped working and the screen went blank.

        I went to the System Administrator (next door) and said "Iain*, my machine has just stopped working".

        "Oh, it shouldn't have."

        "Well, it has."

        Iain checked the command he had just typed.

        "Oh, sorry, my mistake."

        Instead of typing

        #kill -1 1

        he had typed, as root, over rlogin to my machine

        #kill 1

        *His name was Iain, he's guilty, so real name this time.

      3. el_oscuro
        Mushroom

        Re: Background

        When I was a lowly corporal, I had already been through my Pointy-Haired WO and survived without incident. We were running mainframe cycles for supply and finance in West Germany, and I had made some updates to the monthly financial cycle so the JCL jobs would get released automatically at the correct time, reducing errors. I also made some minor changes to the print job, which was submitted after the cycle was complete. Normally these jobs worked without issues and were an afterthought.

        But when the cycle ran the first time with my changes, I was out of town. Of course, something went wrong with the print job, and the operator followed the SOP to fix it. They would have called me but I was out of town. And directly as a result of my changes, the deleted all of the temporary spool files without printing them.

        The books were perfectly balanced and the databases updated, but the client had no output. They wanted it, along with my head on a 6250BPI platter.

        Many hours later after a lot of Fast Dump/Restores, we were able to produce the output. I owed my cow-workers lots of bier after that.

    2. Anonymous Coward Silver badge
      Holmes

      Re: Background

      It's a fairly sure-fire way to tell whether an organisation has suffered an episode of server-identity crises at some point.

      If they have, they'll have the nasty colour scheme in place.

      Yes, systems I control do have that.

      1. YetAnotherLocksmith Silver badge

        Re: Background

        Even if they haven't! My parents have two very visually different versions of windows, depending if they're logged in as "pleb" or "admin". They need the admin password because Windows does stuff often enough that demands it, but the staff have access to the computer too, so passwords are needed.

    3. Doctor Syntax Silver badge

      Re: Background

      It works in user territory as well. Several systems, all running similar backgrounds but against different databases. They were set up with different background images.

    4. Stevie

      Re: Background

      I use Reflections for my terminal sessions. Everyone else, younger and smarter than me, uses putty.

      When I start a session I can pick from a palette of colour schemes I set up. Apparently, that is hard to do on putty. I wouldn't know.

      So when, for example, I have sessions open to Dev, Training, UAT and Production they all get different bg/fg colours. Guess which one gets black on bright red.

      I can also remap the keyboard if some very useful ways. our servers have a naming convention such that a name is a string made up of three character substrings. ssh is a four key op for me, with no yptos, and I have killer carpal tunnel issues. Yesyesyes I have to set up the certs on the servers to avoid needless password retyping but that takes a few minutes once in a blue moon.

      Common file systems across servers means the need to zip all over the place during triage can be simplified. Single keystrokes to the rescue again.

      I've just been informed that I may have to use putty in the future. The software installations guys cannot fathom the degree of pushback they are getting from one person in the department.

      1. Anonymous Coward
        Anonymous Coward

        Re: Background

        As a long-time PuTTY user, I'm not sure why it would be hard to set up different color schemes. Configure it the way you want it, including colors, server name, etc, then save as a "saved session". In the future, double-click the "saved session" you want to use, and it uses the previously-configured settings. I just verified that it does remember background colors this way, and almost certainly all other colors as well.

        1. DS999 Silver badge

          Re: Background

          What you guys are talking about requires it to be properly configured on every client, which isn't going to be reliable as even if you start new people out with that default schema it won't get updated for new servers, they could load their own preferred profile (I'd be mightily annoyed if someone wanted to force me to keep the PuTTY default of white text on black screen) and what if someone doesn't want to use PuTTY or needs to access a server with an iPhone/iPad in a pinch when on call which does not have PuTTY?

          You need to make changes on the server end so client side differences like profiles, SSH software, client device used, etc. won't stop your solution from working.

          You could do it with colors in the prompt I suppose as mentioned earlier in this thread, but what if you hate Linux's tendency to go apeshit vomiting ANSI colors everywhere for normal commands like 'ls' and 'man' and turn off ANSI in your terminal emulator?

          What I did many years ago was have root's profile alias 'reboot' to "echo use reboot-`hostname` to reboot this server" and aliased reboot-`hostname` to /sbin/reboot. OK, there are other dangerous things that could be done on the wrong server, I was just addressing a particular mistake that others on the team had done more than once.

          Even if you don't make the root prompt a different color, you can certainly make it very distinct so it will stand out. Some people may still think `hostname`# instead of `hostname`% is a sufficient clue for a root shell, but experience should have taught everyone by now that it is not.

          1. YetAnotherLocksmith Silver badge

            Re: Background

            That's a very good solution.

          2. Robert Carnegie Silver badge

            Re: Background

            On the question of suppressing ANSI color effects in Linux statements, searching for (ansi color filter) was fruitful. Things like "ansifilter", "strip-ansi", and clever spells in sed appeared. You also can substitute colors with some methods. What I don't know is whether these methods work with interactive processes. But it seems that you can bland out some of what's on the screen by filtering.

            1. DS999 Silver badge

              Re: Background

              You don't need to suppress ANSI colors on non-interactive shells. Try it for yourself, run 'ls' and see the ANSI stuff. Run 'ls | cat' and see that it is gone.

              1. Robert Carnegie Silver badge

                Re: Background

                Ah, clever!

                I like to put optional output from scripts that we write for overnight work - not in Linux as it happens, but what I mean is that many statements are printed about what worked and what didn't, only when the script is run from our development tool, which is sensed as "client application name". Overnight, most of that information isn't logged, because logging gets in the way of error messages, or even loses them because the buffer is full of messages like "Module 35 OK. Module 36 OK." etc.

        2. Stevie

          Re: Background

          I'm not sure either, and I could make it do what I want color-wise but putty doesn't do the keyboard remapping thing and I really need that.

      2. el_oscuro

        Re: Background

        I have considered using different background colours for dev/test/prod/etc but found it doesn't always work - especially if you have to ssh from one to another. And these days we always have to use jump servers so we can't tie the colour to a specific server.

        So changing the prompt color is the way to go.

        1. Stevie

          Re: Background

          You need the colors *especially* if you are sshing from one to the other.

          Because it is important to know which is your "baseline" and which is the one you are going to change.

          I do this all the time.

          When I have the luxury of multiple windows I open them. Then transfers between servers can be scp or sftp but I *know* which server is which.

          So I don't eg restore a production database with test database data by mistake.

  2. diver_dave

    Longest wait

    I PM'd our project to change our call recording system. 500+ users.

    Was updating some permissions and accidentally deleted the source permissions file by uploading a completely blank table.

    Stage 1: Panic.

    Stage 2: Panic some more.

    Stage 3: Try and find backup. No joy.

    Stage 4: Ring supplier, who managed a remote restore.

    Stage 5: Dent wallet badly at project end buying copious amounts of beer for tech who not only restored but forgot to document said restore.

    I do like working with Business Systems.... Lovely company and oh so discreet!

    Should have added, this was a parallel run to old system before switching over.

    1. Anonymous Coward
      Anonymous Coward

      Re: Longest wait

      It's a pleasure to work with most of our customers!

      1. diver_dave

        Re: Longest wait

        Is that you Dave?

        1. arachnoid2

          Re: Longest wait

          Sorry I cant do that Dave.......

  3. Will Godfrey Silver badge
    Facepalm

    It could be worse

    When it's your home setup you just trashed. Especially when you'd just been thinking it really was time you sorted out those stale backups... and the other half has just asked why she can't get onto facebook.

    1. 45RPM Silver badge

      Re: It could be worse

      Can’t get onto Facebook? Sounds to me like you’d done her a favour. A Facebook outage is a significant security upgrade!

      1. Anonymous Coward
        Anonymous Coward

        Re: It could be worse

        A Facebook outage can be grounds for divorce north of Watford.

        1. Muppet Boss

          Re: It could be worse

          “What's ‘outage’?”

          “I'm asking you out.”

        2. John Brown (no body) Silver badge

          Re: It could be worse

          "A Facebook outage can be grounds for divorce north of Watford."

          Because south of Watford it's Instafram influencers and other wannbees?

    2. Pascal Monett Silver badge

      Re: It could be worse

      I trash my home system all the time. I fear nothing now. At least, not since I learned the true value of the word "backup".

      It happened one day, way back when, and I was making a copy of my company mail file to my home system. It's just that I had forgotten that the file name was the same as my home mail. When faced with the "Do you want to replace" popup, I clicked on Yes, of course.

      Then it dawned on me. I had just erased my home mail, with all my contacts and years of correspondence - including a response from Ian McKellen himself concerning a mail I had sent to him before LotR came out. I had emailed him to tell him how awesome I thought this was going to be, and he had nicely responded to me saying he was indeed thrilled about being in the film. Now that conversation is gone for all time.

      That's the day the word "backup" became important to me.

      1. Sgt_Oddball

        Re: It could be worse

        Been there, done that. Accidentally dropped the wrong raid array with my Windows server install (it came with the server... No backup of the license key or spare media)... Oops.

        On the plus side I'm now much more proficient at Linux off the back of it, every cloud and all that.

        Still have a stick of dead ram on my desk from my fully populated homeserver (it's a pain chasing which is the bad stick when you have 16 to choose from and the front panel diagnostics keeps jumping from one stick to the next) which went bad shortly after the array fiasco too.

        But yes, there's nothing quite like having people pressure you unnecessarily when you're juggling live and test. Especially on the same box.

        And yes, I've also worked with different colour schemes for live and test/dev. It's a mistake you only make once.

    3. Anonymous Coward
      Anonymous Coward

      Re: It could be worse

      Oh - don't go there.

      My SO is a terror user from hell.

      Some past "incidents":

      - The Internet isn't working. You're an engineer - fix it! (Line down for some reason).

      - The Internet isn't working. You're an engineer - fix it now! (DNS issues out in the Internet).

      - The TV isn't connecting. Fix it! (It was working fine - the program she wanted to watch wasn't even on, though how that ended up being related to the Internet not working.....).

      - My computer is running slow. (It was actually fine, it was a slow loading web page).

      If only I could charge a callout rate for every time I get bugged.... :)

      1. MiguelC Silver badge

        Re: It could be worse

        Well, whenever anything IT/electronic related starts to go south for her, my SO starts huffing and puffing louder and louder until I eventually 'volunteer' to help... and only *then* it is my fault if it's not working :)

      2. Mooseman Silver badge

        Re: It could be worse

        - The Internet isn't working. You're an engineer - fix it!

        Ah, your SO is the same as mine then.

        1. Alan Brown Silver badge

          Re: It could be worse

          as far as family know. I'm a piano player in a brothel

          1. Wexford

            Re: It could be worse

            Well, that's sure something you don't want your old mum to call you for assistance with.

          2. Anonymous Coward
            Anonymous Coward

            Re: It could be worse

            I right RESPECTABLE position compared to those cable-divers! :)

  4. 45RPM Silver badge

    A long, long, long, long time ago, when we were just beginning to get PC based Linux and Windows NT based servers, we still had some hefty old iron based on ancient DEC platforms. Sure, plenty of Vaxen (because everyone did), but also old PDPs. As a noob manager, I had responsibility for a rack of mixed kit in the Development Data Centre. The rack had a PDP-11 in it running - oh bloody hell. The mists of time obscure what it was running - I can’t remember. Suffice it to say, that I was one of the few developers there who understood the software, and who could boot the bloody thing up (although there was a better that 50/50 chance that any attempt at booting would result in failure - so it was a bit of a shenanigan and a bit of a pain in the arse).

    One of the developers who reported to me was a real bright spark (genuinely), but also freshly minted from University. He was developing a genuinely cutting edge platform on a Linux foundation, and it was in its earliest earliest days with some gnarly bugs in it. One gnarly bug required the system to be powered off and rebooted. I mean. No problem there, surely? A PC and a PDP don’t even look alike.

    I was upstairs, in the offices, happily beavering away on my VT when… loss of connection. The PDP had disappeared off the face of the earth, taking the code I was writing with it (not too much though - I saved regularly). I ran downstairs to discover noob-report looking sheepish. For some reason which I still can’t fathom, but might have something to do with it being very late and well past beer o’clock (his choice to stay and finish some work, not an instruction from me), he’d decided to power off the entire rack rather than just the server that we was working on. His server was up and running again, but he didn’t know how to boot the PDP.

    He watched with increasing concern as I toggled in the bootstrap on the switch panel on the front, the PDP began to boot… and then failed. Again, and again, and again. Capricous bloody computer. With each failure, I got more and more angry to the point that, noob-report admitted to me over a beer in the pub after the PDP eventually started, he thought he was in danger of being subject to a walloping administered by myself. Which would have been a remarkable turn of events since I’ve never hit anyone in my life.

  5. Anonymous Coward
    Anonymous Coward

    Hands up, intentional mixup

    Periodically, as I complain about a lack of backup, monitoring, process etc, I've seen techies deliberately shutdown/reboot important servers just to get management to pay attention to the need for failover etc.

    Techies shouldn't need to do it, but sadly even now we have to.....

    There was always something *very* satisfying about deliberately seeing it done though..... it's ability to focus managerial types on the important things is magnificent!!

    1. Anonymous Coward
      Anonymous Coward

      Re: Hands up, intentional mixup

      Unleash the Chaos Monkey!

      https://netflix.github.io/chaosmonkey/

    2. Doctor Syntax Silver badge

      Re: Hands up, intentional mixup

      I've always thought a good response to bean counters wanting to put a figure on the value of some system would be "Shall we switch it off and find out?"

      1. irrelevant

        Re: Hands up, intentional mixup

        Ah... The time the bean counters at one client decided to have a purge of all the unused telephone lines coming into the building. Unused determined by "having no call charges."

        All the incoming-only fax lines for each department, top floor executives, etc., I would have thought could have been forseen. The kicker was their killing off the ISDN line that was the sole Internet connection for the company. (1990s, so mostly just used for email; only the higher-ups had web access.) It was on some bundle tariff so there were no individual calls listed.

    3. Muppet Boss
      FAIL

      Re: Hands up, intentional mixup

      >it's ability to focus managerial types on the important things is magnificent!!

      Certain managerial types can become too focused when deadlines, bonuses etc are involved...

      Around 15 years ago we were performing system upgrades in a financial organization as mandated by the internal audit. A part of this upgrade was a NetApp storage appliance with around 10-20TB of data fronted by a Windows server and used by literally every department of the said organization. It was not business critical on paper but the audit revealed that a lot of business critical processes were in fact using it.

      The amount of files was so large and the NTFS structure so convoluted that in the attempts to back up the data standard copying was simply failing and even more low-level Robocopy reported it would take weeks if not months to complete. Needless to say, regular backups were not an option even before.

      Of course, when the cold, sleeping data was disturbed, it did not take it well and one disk promptly failed leaving the system in a degraded state. Taking a full copy of data from the degraded storage system would then literally take months if ever. A sane approach would be to fix it first before moving any further but the IT infrastructure boss was pressing hard because he had his deadline and his boss had his deadline etc. Being technically literate, he even went as far as to ordering the team to override NetApp's foolproof safety protections via the serial console and proceed with the upgrade.

      Somewhere around this moment it became a big pressing question as to whom the blame will be put on should the charge of the light brigade somehow fail. NetApp was finally contacted for the advice and their engineer came up with the following response, 'I am not sure that the customer fully understands what they are doing'. Needless to say, all engineering efforts switched from completing the upgrade successfully to not being around the fallout zone when the worst happens. Fortunately, the management sensed that their chairs might become shaky too and NetApp was paid to take risks (as in, if the vendor failed, no one could have done it), perform the fix and upgrade properly and with enough time, which they did.

      1. tip pc Silver badge

        Re: Hands up, intentional mixup

        The old blame game, lots of decisions ultimately come down to that.

        Going through one right now, no one else seems to see it though, it’s crystal clear to me and I’ve given my warnings verbally and emails. Don’t blame me when you can’t do it on time.

        1. eldel

          Re: Hands up, intentional mixup

          Email (and it's associated audit trail) are a lifesaver and the reason that I refuse to do anything on a verbal/zoom/chat instruction without an email confirmation that "yes, as per your instruction/request/recommendation, I will be doing x". I'm not the sharpest knife in the drawer but even I learn eventually.

          1. Soruk

            Re: Hands up, intentional mixup

            At the minimum I will take a screenshot of the chat and send it to the other party as confirmation of the instruction.

          2. Muppet Boss
            Pint

            Re: Hands up, intentional mixup

            >Email (and it's associated audit trail) are a lifesaver...

            That particular boss was a seasoned ... manager who would have thousands of unread email messages (when I saw it myself it was 7'000+) and would only selectively open 'safe ones'. His general line of defence was that he was not properly informed (meaning he did not see your email and did not remember you telling him). Sure, assigning tasks is a different matter but that's what team meetings are for ;). There is many a dark path of avoiding the audit trail. Fair to say he was equally skilled in protecting those under his management after genuine ... hiccups.

  6. Andy Non Silver badge
    Facepalm

    A couple of decades ago

    I wanted to check how vulnerable one of the company's public facing applications was to SQL injection and issued a drop table command embedded in user input on the "test system". I must have been distracted or otherwise brain not engaged as protests all around the office erupted as I trashed the live system. Oops. Thankfully I just got a bollocking, not a P45.

    1. YetAnotherLocksmith Silver badge

      Re: A couple of decades ago

      Returning "1" is safer!

  7. Steve Kerr

    Investment bank production settlements power off

    Not me fortunately and many years ago, late 90's

    Tech workinng on some HP-UX UAT system, for whatever reason, he wanted to reboot or power-off/on some multi-cabinet HP-UX server.

    He had major visual impairment, he had these huge monitors with very large font sizes and he read from a distance of about 4 inches.

    He decided to go up a floor to the computer room and past the ops to the HP-UX server and flipped the power button, except..... he powered off the production settlements server during the business day.

    Don't quite remember how long it was down for before it was got up running again and a lot of shouting happened.

    Access rights for tech people were rescinded soon after, tickets had to be raised with th Ops people to gain access and then they let you in,

    Saying that, it was a big room, easy to get lost in and every corner looked the same, always fun finding BT people and others who were lost in there and pleading for help on finding the exit, internal people too - there were 1000 full height racks in there.

    1. Shadow Systems

      At Steve Kerr, re: getting lost.

      Take a common drink coaster & print a small map of the server room on each side, laminate it with clear box tape, & get a few thousand copies made at your local printing house. Leave them in the break rooms, the front desk, and at every station in said server room. Then finding your way out is as simple as snagging a coaster off the nearest flat surface & giving it a quick look.

      *Hands you a pint*

      Enjoy a pint & don't forget to keep your (towel/coaster) handy. =-)

  8. Doctor Syntax Silver badge

    The person who deserved the real bollocking was the mico-managing manager. Very likely, however, she didn't even recognise it as a learning opportunity, at least, not for her.

  9. Sequin

    Having to support servers around the world (UK, US, Germany, Australia, Pakistan) I have learnt after several bottom-clenching mistakes, I now change the colour schemes on each of my remote desktops. I set a desktop wallpaper which includes the country's flag, with the server name in large text underneath, with a background colour of red for live systems, yellow for test systems, and green for dev systems. This simple scheme has saved my job on several occasions!

    1. Peter2 Silver badge

      A tool I think that you really need is BGInfo; free download from Microsoft as part of the Sysinternals suite.

      https://docs.microsoft.com/en-us/sysinternals/downloads/bginfo

      Force it to run via group policy on servers, and problem solved.

  10. Kubla Cant

    Confused

    a large motorcycle broker

    I read through most of this article wondering what a motorcycle broker does. In my days as a motorbike rider I broke more than one, but that's obviously not what "broker" means. I'd always assumed that the supply chain for motorbikes is much the same as any other pricey consumer good, but this seemed to imply that dealers sourced bikes from some kind of intermediary market-maker.

    Towards the end it becomes clear that it's an insurance broker.

    1. Muppet Boss
      Pint

      Re: Confused

      >a large motorcycle broker

      Like a burly guy on a large cruiser? Cheers mate, that was funny.

      P.S. Talking about insurance brokers, at first, I was like, why is Kubla Cant promoting Confused? And then, ah, Kubla Cant just got confused. See title. ;)

  11. ChrisC Silver badge

    "The reboot meant every client of ours was uninsured for the eight minutes it took to reboot the server."

    By the sounds of it, the design of the policy update process means that, even in the absence of any unexpected server reboots, all of their clients would regularly suffer periods of being "uninsured" (as in, *appearing* to be uninsured due to their policy not showing up on MID, though not *actually* uninsured through not having a policy in the first place) for however long it took to upload the new data having first wiped the old data each time the update process was run.

    1. Tom Chiverton 1

      No, because transactions

      1. J.G.Harston Silver badge

        In another life I dealt with taxi licenses. We had one driver who would buy insurance daily by posting some money through the letterbox of his broker before starting a day's work. We had to explain to him, slowly and using short words, that no he wasn't insured. Hurray for online insurance broking.

        1. Dave314159ggggdffsdds Silver badge

          Are you describing it badly? There's no reason an insurance reseller couldn't operate on that basis.

          1. J.G.Harston Silver badge

            Not when the driver goes on shift at 7am and the broker doesn't open the office and pick up the post until 9am. You can't get insurance "starting two hours ago".

  12. Anonymous Coward
    Anonymous Coward

    ...she saw everything as my fault

    Reminds me of at least two of my bosses over the years.

    The last one had a habit of accusing you of something. You'd provide proof that it wasn't you, even to the point of identifying specifically who it was (even though you really shouldn't have had to be doing any of that).

    But as we know, PHBs are never wrong, and proving them to be so just stirs up the hornets' nests of their egos.

    So he'd reluctantly admit he had been given incorrect information... and then proceed to behave exactly as if he'd been right for the next couple of months. It would even come up in your annual appraisal.

    1. Dave314159ggggdffsdds Silver badge

      Re: ...she saw everything as my fault

      That's called bullying/unfair discrimination and you should raise a formal complaint.

      1. Anonymous Coward
        Anonymous Coward

        Re: ...she saw everything as my fault

        nah its called being a c**t

    2. Terry 6 Silver badge

      Re: ...she saw everything as my fault

      I've had that one.

      Manager; "You made such and such error in September"

      Me; "No, we established at the time that I'd correctly followed the required procedure. There was an error downstream"

      Manager; "But there was such and such consequence because of what you did.."

      Me; "I did the only permitted thing"

      Manager; "But you should have not done that"....

      1. Anonymous Coward
        Anonymous Coward

        Re: ...she saw everything as my fault

        Me; "f*$k off"

  13. chivo243 Silver badge
    Thumb Up

    Saw it coming!

    I never use full screen for RDP. RDP for macOS shows the name of the server you gave to the connection. BGinfo is also deployed for a second line of defense against this boo-boo!

  14. BinkyTheMagicPaperclip Silver badge

    This is why multiple desktops are a good idea

    Local servers on the main desktop, production on another, UAT on a third if you really need to be certain.

    It's a pity the Windows 10 multiple desktops don't allow different wallpaper or background colours per desktop.

    However, the Sysinternals Desktops app does. The only disadvantage is that because they're distinct desktop objects, it's impossible to drag programs from one desktop to another. It also seems to be a bit unhappy on Windows 10, the explorer bar doesn't work properly. Bginfo can be used to set a different background.

    1. yetanotheraoc Silver badge

      Re: This is why multiple desktops are a good idea

      "it's impossible to drag programs from one desktop to another"

      Surely that's by design.

      1. BinkyTheMagicPaperclip Silver badge

        Re: This is why multiple desktops are a good idea

        Yes, it's absolutely due to the architecture of multiple desktop objects, but window dragging between desktops is generally expected in almost every other multi desktop product.

        1. Anonymous Coward
          Anonymous Coward

          Re: This is why multiple desktops are a good idea

          You can copy/paste files including portable applications between RDP sessions on all currently supported versions of Windows. Just right click -> copy on source then right click -> paste on destination (or CTRL C - CTRL V). Just don't try it with huge files on WAN links. Obviously won't allow installed programs with dependencies to be copied to a new session.

          If you are running a recent version of Citrix VAD you can drag and drop some files straight into a session. Microsoft often implement new Citrix functionality into RDP a couple of versions later so it might come to RDP in a few years time.

  15. Vometia Munro Silver badge

    When the computer you are working on is not the computer you are working on

    Post OS upgrade, I finished off some bits and pieces using the vt220 compatible sat on the minicomputer and turned the reset key. While the key was in the "full reset, yeah you asked for it lol" position I noticed the label on the computer didn't match the name of the system I was working on. And remembered the point I raised myself a few days previously that the new policy of putting consoles on any available surface was a bad idea in itself, but an especially bad idea when "any available surface" included typically unrelated minicomputers whose top was a convenient desk height.

    So yeah, having raised this point myself, and having a head full of filesystems, software versions, if the various patches and updates had been invited along to frolic and so on I'd subsequently forgotten that minor point. The computer, the computer. Oh yeah, that thing the console is sat on. Computer! Reset key! Final stage of this tedious experience which my coffee-deprived mid-afternoon crash just thought "thank all the expletives in the universe that is over". And then the realisation. The computer; well it's a computer, but not The Computer. Not the one I am working on. If it is The Compuer, it's... oh bugger, the one that the "Death Row" executive offices use. The executives themselves didn't partake, I mean this was the early '90s and keyboards were women's work, but their PAs were quite fearsome. So, er, sorry about that, couple of dozen high-power and probably now quite grumpy senior people whose work I've just nuked.

    The fallout was surprisingly mild. I was sheepish about it and my admission may have got about as far as a rather high-pitched <eep>, the Most Senior Of All PAs seeming actually quite relaxed and "well the computers normally work all the time, these things happen" and while I'll never be entirely sure if she knew "these things" were due to the sleep-deprived figure in front of her having a bad day and inadvertently blundering about and causing chaos... oh, who am I kidding, she knew and was gracious enough to let me go on my way without further ado. No complaints were filed, or my manager did a very good job of composting them. The wayward console problem was solved soon afterwards using a curious little RS232-based LAN controlled by a COBOL application on a 286 PC. Sounds a bit ancient but worked really well.

    I didn't reset or otherwise crash any more machines but I did have various other escapades and misadventures, largely because I was in my early 20s.

  16. Anonymous Coward
    Anonymous Coward

    "It's the database the police use" ...

    And ? Big deal ?

    There have been many news reports over the years where an un-updated MID has caused some poor sod to have their car seized (with them on the hook for towing and storage and recovery) with not a jot of shame from insurers.

    (The police always have the excuse "Computer says no" as their get-out).

    And it's the motorists responsibility to ensure the MID is up to date *before* they set off.

    https://ownvehicle.askmid.com/

    1. YetAnotherLocksmith Silver badge

      Re: "It's the database the police use" ...

      The database *was* up to date when the driver left. And since it is illegal to use your computer while driving, and stopping on a motorway would cause bigger issues, the law, as ever, is an ass.

      1. Dave314159ggggdffsdds Silver badge

        Re: "It's the database the police use" ...

        As usual, it isn't the law that's an ass. It's the people making ludicrous/nonsensical claims about the law who are asses.

        I know it can be tempting to uncritically accept things that fit your preconceptions, but you even pointed out the absurdity here! Surely it's obvious that it's more likely someone is wrong on the internet than that the law is utterly absurd?

    2. Dave314159ggggdffsdds Silver badge

      Re: "It's the database the police use" ...

      "an un-updated MID has caused some poor sod to have their car seized (with them on the hook for towing and storage and recovery)"

      You've imagined that.

      "And it's the motorists responsibility to ensure the MID is up to date *before* they set off."

      Nonsense. From your own link:

      "this check on askMID is not proof of insurance nor is it proof of no insurance but confirms only if the vehicle is currently showing on MID."

      There is a legal requirement to have insurance. There is no legal requirement (for a driver) to ensure MID is correct, although obviously doing so can stop you getting pulled over due to errors.

      I've been pulled over for no insurance a few minutes after buying insurance. The police were quite happy to check other things for proof when I told them the timescale; they certainly didn't reify MID.

      1. irrelevant

        Re: "It's the database the police use" ...

        I actually got a letter from the local police late last year, "your car was spotted driving without insurance. We expect this was a mistake, fix it." or words thereabouts.

        As it happens, it was a mistake, and not even mine. It's a lease car, and they do the insurance, and it seems something had gone wrong when we extended the lease, six months previously.. As it was a weekend I got the letter, and the insurance firm was shut, and now we all knew, I wasn't allowed to drive it. The leasing firm gave us an unlimited taxi account to use until it was sorted out. Nice, if inconvenient..

        I'm still amazed that it both took so long for the cops to spot, what with most of my driving being city-centre pottering about, and that they simply wrote to us rather than stopping us and hauling me away in irons.

  17. Bruce Ordway

    Yes

    Yes, I stopped the wrong database once.

    Only about 5 minutes to recover but...

    I have never forgotten the noise wave as it swept through the building in my direction.

    Since then, I check my screens multiple times before I commit any entries.

  18. amanfromMars 1 Silver badge

    Excellent Foresight, an Avid Fan of Enriching Enlightening Hindsight for Divine Experiences

    You gotta just love and admire and appreciate the way El Reg deals with such problems, with an opportune 10 minute window to right and/or reconcile wrongs and consolidate records with updated versions for programming future programmed events.

    1. WhereAmI?

      Re: Excellent Foresight, an Avid Fan of Enriching Enlightening Hindsight for Divine Experiences

      Dammit - I thought he might be away flying helicopters. No such luck, it seems.

  19. Zolko Silver badge

    rm -rf *

    my then-girlfirend was happily logged in to some remote workstation through her own Linux laptop, and was doing some really important research stuff (she said). But that stuff turned out to not work, so she wanted to remove it and begin again. So she typed the dreaded rm -rf * in the shell of the window to the remote server ... when she realised that the connection had been closed and she was actually again on her laptop, in her home directory.

    Fortunately, this only removes the inodes of the files and doesn't shred the data on the disk, and she could recover most of her data.

    1. Anonymous Coward
      Anonymous Coward

      Re: rm -rf *

      # umrm

      -sh: unrm: command not found

      # man cpio

    2. Martin

      Re: rm -rf *

      I have a suggestion to make re: that one.

      I never ever type "rm -rf *"

      I always go up one directory and type "rm -rf <name of directory to be deleted>"

      Then, if I'm in the wrong place, I get an error saying "<directoryname not found>"

      It's not foolproof - it won't save the "damn, typed that in production instead of test system" issue, but it solves a lot of silly mistakes.

  20. Sparkus

    'twas ever thus...

    rm -r *

  21. Terry 6 Silver badge

    This is when you realise the difference between the good boss ("Stuff happens, don't make a habit of it") and the shit boss ("Heads are gonna roll and it won't be mine").

    Life can still be good when the shit boss you report to is below the good one. Not so much the other way round but knowing which way round it runs tells you all you need to know about the organisation's culture, (And how long both bosses will likely stick around).

  22. Stuart Castle Silver badge

    A few years ago, we needed an inventory tracking system. With a few of the team being good coders, and being unable to find an inventory system within our budget that fulfilled all our needs, we designed our own that was essentially a Java Web Service that acted as interface to an SQL Server Database for a couple of websites and an app that all had various inventory functions.

    This system was in heavy use for months, when suddenly, a couple of the App users reported it was failing. Then both websites failed. After an hour, our DBA was able to look into it. He noticed almost immediately that someone had renamed the primary table the system used as a full stop. When boss tried to find out who had done it, it didn't take long for me to work it out. Apart from the DBA, (who is unlikely to have got involved), three of us had the relevant access to do it. I knew I didn't (I don't even fire up the SQL Server DB Manager software unless I need to do something, and I had no need to do anything that required the software). The 2nd technician who had access wasn't even in work at the time, and while we did have remote access to our machines at the time, he really is unlikely to do anything work related while he wasn't in the office. The third denied it as well. I have no idea whether he did it (although the evidence suggests to me he did). Because he couldn't conclusively prove who did it (Essentially while Technician 2 could prove he didn't, being off work, neither Technician 3 nor I could prove we didn't or the the other did). I'm certain the DBA could check logs, but that didn't happen. So our manager did the only thing he could. He asked the DBA to repair the damage and remove our access to change the Database structure. This is fair enough - the system had matured to the point where we didn't need to change this anyway. Arguably, due to the work we were doing, Technician 2 and I probably needed database access more than Technician 3, but Technician 3 is the only one who complained, which makes me wonder what he was doing that he couldn't do through either of the websites or the app.

    1. YetAnotherLocksmith Silver badge

      Illegally renaming tables, by the sounds of it!

  23. irrelevant

    Reboot

    Back in the mists of time, I was working as a programmer at a reseller of accounts software. This was based on a varient of COBOL and ran multi-user on comodity hardware. (the OS was "BOS"..) you could easily support a couple of dozen users on a 286, assuming you'd bolted on enough serial ports - everything ran on serial dumb terminals. Mostly Wyse.

    We'd just got this new trainee in at the office, and for lack of anywhere else to sit him, had plonked him at the console of the box running the dev dept. It offered the exact same user interface as the terminals, but usually wasn't used except when somebody needed access to the floppy or tape drive and didn't want to walk across the room twice.

    Anyway, we were all beavering away when every terminal in the room froze.. Looking towards the "server", it transpired this new hire had got himself stuck, and spotting he was sat "at a pc" had simply hit the reset or power switch to reboot. We had to educate him gently into the implications of what a "multi-user" system was... I'm fairly sure he didn't do it again, though this might be been achieved by escalating provision of an extra dumb terminal!

    1. Jou (Mxyzptlk) Silver badge

      Re: Reboot

      You don't put noobs so close to the server. You've learned that lesson back then, I guess :D.

    2. Wexford

      Re: Reboot

      I did this very thing myself on the first day at a new job (my first actual job!) in the mid 90s. I was sat down at a desk with a grey pizza box and a monitor on it. Didn't look like it was switched on so I reached around the back and felt a power button, which I duly pressed. Nothing came on the monitor, and I didn't investigate further as the boss in the next office called out for me to come and say hello to another team member.

      While I was next door, a flustered colleague went running past and into my office, then came back and complained that someone had turned the file server off. "That's terrible," I said, and resumed my conversation with the boss and colleague who were telling me that I'd have a new Windows NT 3.1 PC ready to go on my desk in an hour or so.

      No idea why they had a file server sitting on the new kid's desk, but there you go.

  24. Anonymous Coward
    Anonymous Coward

    Something similar happened here, just last week. Luckily (?) it was outside "consultants" who did it, not me.

    We've got an outside group who are installing some sort of HA/Failover software on our AS/400. I'm just a network guy, so I stay far away from that strange mojo machine. Anyhow, one day last week, the alerts start coming in that the AS/400 was offline. Come to find out, one of the Consultants had two 5250 screens open - one to our server, one to another customer's server. He was supposed to reboot the other customer's box, but got his terminals confused and, well, you can figure out the rest. This was at 2:00 in the afternoon, it wasn't even after-hours.

    Yeah, we're still going forward with this project and these "consultants", as AS/400 sysops are apparently quite hard to find. Good ones are, seemingly, the stuff of dreams.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like