back to article Oh! A surprise tour of the data centre! You shouldn't have. No, you really shouldn't have

Though the week is over, for some the weekend does not involve frolics and adult beverages but a nerve-wracking 48 hours of watching the company phone. Welcome to On Call. Today's story comes from "Guy" (not his name) who was enjoying his second weekend of being on call. He'd been told that Saturday was the only day when the …

  1. UCAP Silver badge

    I did get a call once to help remotely diagnose why a server was refusing to boot. Took a couple of hours to sort out. Mu company is in the UK, but at that time I was standing outside the Olympic Stadium at Montreal (Canada) which the wife & 2 progs had just gone into to have a look around.

    Never did see the inside of the stadium <sigh>

    1. Pascal Monett Silver badge

      Once I was on holiday in the USA when I got an SMS from a client asking me why I wasn't answering my mail. I had to phone at 6AM (California time) to explain that I was out of the country.

      The interesting thing (from my point of view) was that my associate at the time was also in contact with the client, and he apparently did nothing to handle the issue before they texted me.

      He is no longer my associate.

      1. MiguelC Silver badge

        I once spent an hour and a half on the bottom of a ski slope watching all my mates ski on (initially) pristine snow while I did remote support for a piece o SW I'd had the misfortune of taking over mid-development some weeks earlier... adding insult to injury, when I came back, I spent days knocking on doors until someone at my company "magnanimously" signed-off to pay me back the roaming charges

        Of course none of the other members of my team, who were not on holiday, could take the call...

        1. captain veg Silver badge

          Having been involuntarily held to the premise that "mid-April" meant, in fact, precisely the 15th of that month, I delivered an experimental version of the software which had a file name of <application-name>-<experimental-feature>.exe. And that worked just fine on the then-current Windows XP which my workstation ran.

          Client was running Citrix MentalFrame. It stopped parsing the exe name at the hyphen and passed the rest in as arguments. As it happens, the <experimental-feature> started with the letter A, and my command-line parsing code considered any argument that started with "-a" as a request to show the "About" screen.

          Result, the users of <experimental-feature> on MentalFrame, which was all of them, just saw the About... box.

          This lost me an additional half day on the pistes of Val d'Isere over and above the four days that it had taken to deliver <experimental-feature> and book a flight to Geneva.

          I cannot tell you how much I hate Citrix.

          -A.

    2. Anonymous Coward
      Joke

      > which the wife & 2 progs

      Let me guess: she didn't give birth to them but forked them?

      1. UCAP Silver badge

        progs => sprogs

        <sigh>

        It has not been a good day, so far.

        1. H in The Hague
          Pint

          "It has not been a good day, so far."

          Have one of these -->

          1. UCAP Silver badge

            <gulp> <gulp> <gulp>

            Thanks, I needed that.

        2. Doctor Syntax Silver badge
          Pint

          "It has not been a good day, so far."

          POETS

        3. MachDiamond Silver badge

          "progs => sprogs

          <sigh>

          It has not been a good day, so far."

          For an IT person, the first spelling is appropriate. You could also call them iterations.

      2. spireite Silver badge

        Multithreading in action

        ..... but they sometimes interrupt at the same time

    3. Xalran

      In my case the call was while I was ( in vacation ) having lunch on the Glacier d'Argentière Moraine...

      ( they tried to call me before, but since I was obviously somewhere on the glacier I was out of coverage )

      I ended up spending an hour troubleshooting X.25 by phone, sitting on a rock, with my icepick, iceshoes not far, and with the harness still on trying to eat a sandwich while the people accompanying me were wondering how long it would take.

    4. Stuart Castle Silver badge

      I had a problem that was pretty much the opposite of UCAPs (above)..

      A few years ago, we bought a then state of the art Lexmark A3 colour Laser printer. I was primarily responsible for this (no idea why - while it was flasher than our normal printers, it was just a printer), with another technician helping.

      It had a particular problem where the paper would jam in the Duplexer, and multiple engineers from Lexmark failed to fix the problem. There was a particular trick to getting the paper out of the duplexer and resetting it. Me and my colleague both knew the trick.

      For the first time in *years*, I went on holiday, going to Vegas. When I got back, my boss was in a foul mood, and shouted at me that this printer he'd spent so much money on was not working (he actually hadn't, we provided support for the printer, but it was bought by another department). It had been out of order for a week. The fix was to push a tiny part of the mechanism back into the right position.

      I told him to calm down, and I would look at it. Sure enough, the paper had jammed in the Duplexer. I cleared it, and left the printer clearing the huge backlog of jobs that had built up.

      I asked the technician that supported the printer with me why he hadn't fixed it. He said he wasn't aware there was a problem. He worked for a different department, the one who actually owned the printer. My department supported it on condition our users were allowed to use it.

      I suspect what had happened is our users tried to use it, failed, then complained to my boss, who didn't really investigate, just shouted at me.

      I went to see my boss. I pointed out that there was another technician who could have fixed the problem, but no one bothered to contact him. I also pointed out that the entire department had my mobile number (I had my mobile with me), and in an emergency, they *could* have contacted me, and I would have taken them through the process of fixing it.

      He said "but you were on holiday". I answered, asking why no one had contacted the other technician, and pointed out that while I am happy to carry the can if I have failed in some way, I hadn't failed, I wasn't going to take the blame for someone else's failure. He did sort of apologise..

  2. elsergiovolador Silver badge

    Cliche

    I once got called in to see why one of services is down. They had a brand new server delivered a week before and it was plugged in on a table in developers' room. They were desperate for any extra bit of processing power, so they installed everything before racking it in the server room and they planned to move it on a "quiet" day (hope you catch the irony). Anyways. Friday night, got a text that service is down, go check. Indeed there was no ping, so I went to see what's up.

    Yep, cleaner has disconnected the network cable.

    1. Doctor Syntax Silver badge

      Re: Cliche

      I think it's some sort of special training cleaners get - be sure to unplug everything before you start cleaning.

      1. big_D Silver badge

        Re: Cliche

        Builders as well. Another time, we were having additional AC put into our server room, as it was being expanded to accommodate a couple of new mini computers...

        The builder just strolled up, unplugged one of the microVaxes and plugged in his drill!

        The "real" VAXes were fine, they were hard wired and had a huge, physical switch to power them off. But the microVAXes were using standard 13amp sockets.

        1. My-Handle

          Re: Cliche

          We had a guy come in to service one of the boilers on-site. It just so happens that the small warehouse that housed the boiler also had a building relay link plugged in at an adjacent corner. The nitwit unplugged the thing (ignoring the "DO NOT UNPLUG" label), cutting the other building off from the offices and main internet access point.

          The IT director was up there like a shot and tore the guy a new one.

        2. J. Cook Silver badge

          Re: Cliche

          Installers, too- we had a company installing some new gear in our lab, and they somehow managed to unplug one of the vmware hosts in there, and in the process of plugging it back in, glitched it in such a way that the BMC on it locked up..

          1. The Oncoming Scorn Silver badge
            FAIL

            Re: Cliche

            Installers - Drilling a hole (Carefully) through a glass wall into a state of the art temperature\pressure controlled* laboratory to run a cable, causing all sorts of additional alarms & alerts to start screaming.

            *Door alarm sounds if door is open for > normal transit time.

            1. Anonymous Coward
              Anonymous Coward

              Re: Cliche

              Workman drilling a hole in the concrete ceiling of the computer room - during prime shift. He was standing on a set of steps - alongside the row of 200MB exchangeable disk drives. The drive covers were covered in dust and bits of concrete.

              The site engineer interrupted him - and explained that even cigarette ash was enough to cause a disk failure. The response was "uh uh - brrrrrrrrrr"

      2. Anonymous Coward
        Anonymous Coward

        Re: Cliche

        We had a server plugged into the wall... but... you know that siren that Dabbsy demoed in his column?

        We pretty much had one of those hooked to the server UPS.

        When the guy unplugged it, I think his hair was straight for a week, and his ears were ringing for 2 weeks.

        I also remember a UPS that was basically 100% iron-core transformer. It was the size of a bread box, but it weighed near 100lbs. It had a bright orange "TWO MAN LIFT" near the handle. I remember just about hearing a guy's spine crack when he jogged by and casually tried to pick it up one-handed.

    2. big_D Silver badge

      Re: Cliche

      That is how I learnt to save my work regularly...

      I was writing documentation on a Mac Plus and it was getting late (18:30). I had everything finished and was putting in the index flags into the document - every word on every page that should appear in the index had to be individually flagged! (I would have thought, creating an Index list would have been more sensible, but, no, instead of typing out a thousand words to index, I had to go through 400 pages of documentation and flag each occurrence of each word individually!

      I had been struggling through and saving the document would take a couple of minutes, so I had not saved for a while, to decrease the wait for beer-o-clock...

      Then the cleaner bustled in and phup! My screen went blank!! She'd just yanked out the cable for the Mac, so she could plug in the vacuum cleaner! GAAAAAHHH!

      1. Captain Scarlet
        Devil

        Re: Cliche

        I take it you got a shoulder shrug when you swore at them

        1. Anonymous Coward
          Anonymous Coward

          Re: Cliche

          > I take it you got a shoulder shrug when you swore at them

          My boss got a bit more than a shoulder shrug when he calmly said "You're fired. Get out. Now." to a cleaner that did a similar thing. Didn't raise his voice at all.

  3. DailyLlama

    I got chewed out on a Friday morning once for not answering the on-call phone the previous evening.

    It turned out that the pub I'd gone to do the quiz at was in a dead-spot for phone signal on the network that the on-call phone was using, so it never rang. I filed that information away for further use in case I didn't want to take a call again in future...

    1. big_D Silver badge

      I went on holiday in the mountains. No good reception.

      I always managed to call the secretary once a day to check in.

      When I got back home, I suddenly got an SMS for something that should have been sorted out 2 weeks earlier! I quickly contacted my deputy and asked if the problem had been sorted... Then I got a chewing out from my boss for not checking my phone regularly!

      Getting his secretary to confirm that I had checked in regularly didn't help.

      I now have a much better job - I have to turn the phone off/make it silent and ignore, when I am not working! I have it set up to automatically turn itself silent at 5pm through to 8am.

      1. logicalextreme

        I got reamed (genuinely got a disciplinary) after my first ever weekend on call because I hadn't answered the phone during the wee hours. It was an iPhone and for some reason had a hardware switch on the side that muted it (and happened to be engaged when I was handed the phone). My manager steadfastly refused to believe that a) I'd never so much as touched an iPhone before in my life, b) I'd set the phone volume to maximum as soon as I got it, and c) I had no idea that a volume bar displayed on the screen showing max volume might in fact mean that the phone was going to make no noise.

        1. Antron Argaiv Silver badge
          Thumb Up

          Hopefully you were able to trade it (the manager) in for a non-idiot manager

      2. Stoneshop
        Windows

        Similarly

        One weekend, way back, I was assigned to be on call (although that was not in my contract), for the DBA to if he ran into problems with a large and critical database something. DBA had remote access, I did not. I would also be busy that Saturday preparing a moto trials event, and would be camping out on the terrain to the odd moped youth from getting themselves and the non-stops into disarray. But I had my phone charged and on me, and of bloody course the DBA called at about ten o'clock in the evening. So, sitting at a small campfire (it was late October) I guided him through the more intricate incantations of stuffing a database dump into a backup saveset, then on to another system to free up space, after which he could get on with whatever he needed to inflict on the database. Took about an hour in all.

        So far so good.

        Coming into work Monday I was immediately pounced upon by a philosopher who had managed to become head of the IT 'department' (just five people, him included), demanding to know why I had been unreachable that weekend during a most urgent database something. Now, being on the phone with the DBA might have resulted in not being able to simultaneously answer other calls, but there had actually been no other calls at all, before, during or after. So my best guess was (and still is) that nITwIT philosopher just had the wrong number for me, and just mashed 'redial' every time instead of looking it up. But he insisted the number was right , so mine must have been unright which obviously was my fault.

        Well, end of contract then. No big loss.

    2. MrNigel
      Pint

      Wired remote call outs

      Back in the 70's I worked in Leeds Westgate telephone exchange. To allow for Friday afternoon remote pub working the GPO engineers just extended the SFC phone to the Highland pub next door via a reel of jumper wire. This pub was used in a few episodes of Frost and (allegedly) the phone can be seen in one of the scenes.

    3. MrDamage Silver badge

      Chewed out because I was on holiday

      Booked 2 weeks holidays, first day Monday, last day Friday 2 weeks later. Brainiac of a manager decided that seeing I was going to be "back after the Friday", I could take the weekend on call shift, right? Except I flew back into town mid Sunday, 2 days after I was supposed to go into the office to pick up the on-call phone.

      Boss chewed me out for not taking the shift I was assigned, and gave a written warning. Took it up with HR and the IT GM, 2 weeks later we had a new manager, and this guy was reassigned to manage the mail room.

      1. A.P. Veening Silver badge

        Re: Chewed out because I was on holiday

        and this guy was reassigned to manage the mail room.

        Judging by his reported performance still way above his level.

  4. Evil Auditor Silver badge

    One Christmas feast with the family and that poor IT sod got a phone call from a user who noticed a CryptoLocker taking over. Only months later, the IT was fully functional again. And with such incidents, I'm rather glad not to be on the operation side anymore.

    1. Anonymous Coward
      Anonymous Coward

      I was called out on Boxing Day 2015 after the floodwaters in Leeds entered the datacentre for one of the telephony providers for the customer (ironically the government department that deals with flooding and other environment issues). Cue 8 hours on conference calls agreeing that we couldn't do anything until the telephony was sorted... But it was at double-time and it got me out of the sales :)

  5. Ashto5

    Gotcha

    I was called in to look at why the warehouse was not running and all of the warehouse computers were showing a critical system error dialogue.

    Traced it to the existence of a critical error file in a directory, deleted the file the warehouse came back online.

    {takes hero stance}

    I had to do it because the usual guy was away on holiday.

    Looking further into the issue I found a little program running on a timer that created said file at randomised times good few days between creations.

    Turns out you get paid extra for being on call and being called and mucho gravitas by getting the warehouse back up and running, so El Sid had written a batch program that sat there and created the “incident”.

    Now that’s mortgage driven development.

    1. Evil Auditor Silver badge
      Thumb Up

      Re: Gotcha

      «Now that’s mortgage driven development.»

      And, I hope, development driven dismissal. Similar used to happen in the 90s over here. That is, until the remuneration package was changed from getting paid for being called to getting paid for being on call. All of a sudden the number of incidents dropped considerably.

      1. Wally Dug

        Re: Gotcha

        Not sure how truthful he was being, but when I worked in a bank, a colleague told me how his flat overlooked the busy - and therefore important - ATM of the branch. He would regularly insert his library card into the card reader, go back home and await the on-call phone call five minutes later. Yes, he was paid by the call-out.

        1. John Brown (no body) Silver badge

          Re: Gotcha

          Must have been the early days before they put cameras in them :-)

      2. Anonymous Coward
        Anonymous Coward

        Re: Gotcha

        At a previous PlaceOfWork a different department solved all such needs for us. Turns out that if one use the same somewhat under-specced windows server for both production and development it will tend to hang at random times, requiring a tech to take a trip to the data centre to reboot the poor thing (this was over a score of years ago).

    2. juice

      Re: Gotcha

      I knew someone once, who was looking for a new job, and had a job interview arranged. Unfortunately, they were also short on leave and unsurprisingly, didn't really want to discuss their itchy feet with their manager.

      Fortunately, their company also had a call-out policy, where if you were called out in the wee hours, you didn't have to come in the next morning.

      One little cron script later, and they were able to attend said interview...

  6. ColinPa

    year 2000

    Year 2000 support. I wasn't called out, but I had to go to the US on Jan 1st - to provide support when the US banks opened.

    Jan 2nd I had an email from my wife. Something went "ping" at midnight - what is is?

    Jan 3rd another email from my unhappy wife. WHAT IS GOING PING - as it wakes her up. She was tired from doing support.

    Jan 4th email from my very unhappy wife. She had taken contents of my bedside table and spread them on the bed and floor and sat up till the witching hour.

    We had all been given pagers for the year 2K support - just in case. I had thrown it in the drawer besides my bed and forgotten about it.

    At midnight all the pagers were sent a message like "the lottery numbers" etc.

    She took the batteries out - and it took a long time for her to forget.

    1. Stephen Wilkinson
      Joke

      Re: year 2000

      I bet she hasn't!

    2. I ain't Spartacus Gold badge
      Happy

      Re: year 2000

      and it took a long time for her to forget.

      Estimated timeline:

      paperless office

      self-driving cars

      fusion power

      ColinPa finally forgiven by wife

      heat-death of universe

      ...although possibly the last two items are in the wrong order...

      1. The Oncoming Scorn Silver badge
        WTF?

        Re: year 2000

        It scared the fuck out of me that the Happily Ex-Mrs Oncoming Scorn, could throw back at me something I had apparently said, 9 years, 6 months, 3 weeks, 5 days, 14 hours, 33, minutes & 20 seconds in the past.

        Elephants never forget & are share a memory map closely related to SO's of the female variety.

        1. MachDiamond Silver badge

          Re: year 2000

          "could throw back at me something I had apparently said, 9 years, 6 months, 3 weeks, 5 days, 14 hours, 33, minutes & 20 seconds in the past."

          ....and if you are a forgetful bstage that errs more than occasionally, you have to change SO's periodically as the inventory continues to grow with each infraction. After a while it's just too much to deal with. Better to upgrade to a newer model and start fresh again. Lather, Rinse, Repeat.

          1. stiine Silver badge

            Re: year 2000

            That's too much trouble, besides, where would I find another who would put up with me in the first place?

        2. Snapper

          Re: year 2000

          I'm sure they have a different compartment in their brains specifically for that, as my B&C cannot remember the plot of the simplest TV 'drama' (women crying + shouting + banged doors) after the second advert break!

          She can remember an argument from 30 years ago as though it was yesterday.

          1. Swarthy

            Re: year 2000

            Total Grievance Recall

        3. Diez66

          Re: year 2000

          Mine remembers things that didn't happen, or she/I didn't do; ouch.

    3. Flightmode

      Re: year 2000

      Reminds me of the time, must have been right before the year 2000, that I woke up around three in the morning, realizing I'd heard a short, faint and muffled BUZZ a number of times in my sleep. It was infrequent enough that you couldn't easily spot where it came from or count the number of seconds between BUZZes, perhaps a minute or two; but once you'd heard it you couldn't unhear or ignore it. My wife, also awakened (by the BUZZ or by me, I can't remember), and I tried triangulating the sound by moving around the apartment, listening intently and ruling out where it WASN'T coming from. (Naturally one of us would without fail say something like "I think it sto-", just as it BUZZed again, wasting a cycle.)

      Eventually we did find the source. It was my old pager that I hadn't used in a couple of years, complaining about low battery (a single AAA if I recall correctly). It was in my nightstand drawer, literally 10 cm away from my pillow. Must have taken at least an hour to find.

      1. Stoneshop
        Devil

        Re: year 2000

        Reminds me of the time, must have been right before the year 2000, that I woke up around three in the morning, realizing I'd heard a short, faint and muffled BUZZ a number of times in my sleep.

        Ah, an annoy-a-tron. Smoke alarms are also in that class: single short beep with several-minute intervals to indicate a low battery, and due to the pitch and volume of that beep it's hard to pinpoint. And of course those units don't then blink a LED, as that would take valuable energy.

        I once built an actual annoy-a-tron, an electronic circuit that would generate the sound of a mosquito a couple of minutes after the light was switched off in the room it was hidden in. Light on it went silent immediately. The room in question being one of my fellow students' who had a couple of annoying habits himself, so this revenge was justified.

        1. Anonymous South African Coward Bronze badge
          Devil

          Re: year 2000

          You, sir, should be in line to win a BOFH awar<KZERRRRRT>

        2. Flightmode

          Re: year 2000

          I built a similar-but-different circuit after reading about the Annoy-atron; I called it the IR-ritator! It was roughly the size of a postage stamp, about a centimeter thick including battery. (PCB courtesy of OSHpark!)

          At the time I was working in an office where a bunch of people had multiple TVs and Set Top Boxes on their desks and on test benches, meaning that the office had literally hundreds of remote controls being used all the time. The gizmo had an IR sensor (hence the name) that detected incoming signals and on every thousandth trigger it would play the Nokia tune from a tiny piezo speaker.

          1. Martin-R
            Pint

            Re: year 2000

            Oh that's just evil :-) Have a ====>

          2. stiine Silver badge
            Pint

            Re: year 2000

            You evil bastard...

            And as I had that thought, i scrolled down one line and read the same, but I think you're evil enough that I should tell you again.

        3. John Riddoch

          Re: year 2000

          Oh, smoke alarms... first time I had one in my flat, it took me weeks to track down because it was beeping hours apart and it was only because it finally beeped while I was walking under it I twigged where it was coming from. New battery, problem solved...

          Once you've had that happen, you at least learn that "random beep = new battery in smoke alarm", but the first time is a bugger.

          1. Tim99 Silver badge
            Facepalm

            Re: year 2000

            We have two fire/smoke alarms in a long corridor going from an office to the bedroom. One was a bog-standard americium-241 ionization smoke detector with a 9V battery, the other was a photoelectric (sealed for life) wired to the mains supply. In the middle of the night (when the air got cold?) we were woken by a high pitch failing battery signal. The sound stopped when I was investigating the source. I found a new 9V battery and fitted it to the ionisation detector, because that was "obviously" the one that needed changing. I tested the new battery and all seemed well. The next night it happened again, I didn’t have a replacement battery, so I got out the stepladder and then noise stopped - I removed the "new" one anyway to avoid being woken again. The next day I bought two replacement batteries. When I came home the noise had started again. Obviously the photoelectric was the source, when I looked at specs, the "sealed for life", contained a backup battery with a shelf life of 10 years - The system was installed 7 years before, and required an electrician to remove the old one and replace it. I assume that air movement or heat from opening the bedroom door was just enough to turn the detector warning off…

          2. Stoneshop
            Devil

            Re: year 2000

            but the first time is a bugger.

            Especially if that beep is more of a metallic blip, like one of those clicker toys.

            The people who design smoke alarms to signal battery depletion to sound like that should be locked in an echo chamber for a week, with several dozen of smoke alarms spread around a ceiling just too high to reach, batteries nearly dead.

          3. Anonymous Coward
            Anonymous Coward

            Re: year 2000

            My three storey house has a smoke alarm on each level of the stairs. Have to find which one is blipping intermittently in the middle of the night - then find something to stand on to disable/fix it..

            Recently there was an unusual noise in the kitchen. Was the central heating pump failing? Was the fridge freezer running continuously? A neighbour running their car outside?

            Eventually it was triangulated with difficulty to the digital clock on the wall - a low battery. Unfortunately I've invested in several of those Youshiko radio synchronised clocks - which are dotted all round the house.

            1. Robert Carnegie Silver badge

              Re: year 2000

              Smoke alarms: I expect that if you press the test button on each one, then the one that's dying will give an unhappy indication.

              Smoke alarms and also clocks: Then, or more or less then, you replace the battery in each one of them, because the ones that aren't dead will be nearly dead at the same time.

              1. Diez66

                Re: year 2000

                I used to buy my alarms a battery for Christmas, wrapped them up the first year because I could.

                That was when they replaceable with 1 year life. Happy days.

        4. Martin
          Happy

          Re: year 2000

          An annoy-a-tron - great name.

          We had one of those recently. Somewhere around our bedroom, there seemed to be a random noise - sounded like someone ran their fingers quickly down a blind or something like that. (We don't have blinds...)

          It came and went. We wondered if it was outside, and some animal jumping on the shed roof? or something else? It would disappear for a few weeks - then suddenly came back, to annoy us again.

          Then one day, I was in the bathroom, and I heard the noise - and caught a movement out of the corner of my eye. I turned my head - the noise had stopped, but an electric toothbrush standing on its end on a glass shelf was wobbling slightly....

          For some reason, the toothbrush would switch itself on briefly then off again - just enough to make the noise.

          It also explained why, occasionally, we'd find the electric toothbrush in the sink - it had obviously buzzed a few times and walked off the shelf into the sink.

          I don't think I would ever have found that if I hadn't been in just the right place at the right time on that day.

          1. Anonymous Coward
            Anonymous Coward

            Re: year 2000

            How about children's toys with all-but-dead batteries? We once wandered the living room for 10 minutes trying to find the source of the faint, deep occasional buzzing noise (like a deep speaker tone, not vibration), to eventually find a toy with batteries with just enough juice to still make some noise, but not its normal one. Can be a creepy sound in a quiet house.

            1. the hatter
              FAIL

              Re: year 2000

              Seems to be a weird failure more in several triggered noise-making toys. I guess at a too-low battery level, it fails to be able to keep a transistor path open, so the noisy bits trigger - but with so little current available it just drones at a low, sinister frequency because that side can't power it's oscillator well enough to get a kid-friendly squeaks and beeps you'd expect. Evidence as to how they knows to start only in the hours of darkness is beyond the explanation of electrical engineering/the laws of physics.

            2. Aussie Doc
              Megaphone

              Re: year 2000

              Years ago son and I were heading to his place (seven hours away) in an even more remote place than I normally lived in at the time.

              In the boot (trunk, I think some of you know it as) was packed a heap of toys for his three kids (my grandmunchkins) including a large battery-operated fire engine.

              For more than half the journey all we could hear from the rear of the vehicle was the ever-slowing sound of 'WEEEEE WAAAAAAW WEEEEEE WAAAAW'.

              It was pouring with rain and the boot was so full it wasn't worth the effort to stop and repack but, boy, what a long drive lol

              Great father/son day nonetheless.

          2. adam 40 Silver badge
            Unhappy

            Upright Electric Toothbrushes

            "Then one day, I was in the bathroom, and I heard the noise - and caught a movement out of the corner of my eye. I turned my head - the noise had stopped, but an electric toothbrush standing on its end on a glass shelf was wobbling slightly....

            For some reason, the toothbrush would switch itself on briefly then off again - just enough to make the noise.

            It also explained why, occasionally, we'd find the electric toothbrush in the sink - it had obviously buzzed a few times and walked off the shelf into the sink."

            Well if you leave an electric toothbrush upright, what happens is all the mouth juice dribbles down through the drive axle over time, and interferes with the electronics inside, eventually causing it to fail.

            Always leave it on its side after shaking out the juice from the brush head, or better still, head down.

        5. TomPhan

          Re: year 2000

          Think Geek, when it existed and wasn't just tie-ins, used to sell Annoy-A-Trons for that very purpose.

        6. J. Cook Silver badge

          Re: year 2000

          Back before ThinkGeek was assimilated by Hot Topic, I had bought both the V1 and V2 annoy-a-trons from them. I deployed them at work, and then after a day UN-deployed them.

          That beep carries quite far in an open-cube farm...

        7. MachDiamond Silver badge

          Re: year 2000

          "I once built an actual annoy-a-tron, an electronic circuit that would generate the sound of a mosquito a couple of minutes after the light was switched off in the room it was hidden in."

          Using the fact that beeps from a smoke detector are unlocatable, you should have put the mozzie simulator in a smoke detector case. A potential upgrade would be to use multiple small speakers pointing in different directions and fade the audio from one to the next to be even more annoying and harder to track down.

          1. Stoneshop

            Re: year 2000

            Using the fact that beeps from a smoke detector are unlocatable, you should have put the mozzie simulator in a smoke detector case.

            This was early 1980's, autonomous smoke detectors weren't really common back then, and it was built from opamps, a 555, a sprinkling of discrete resistors and caps and an LDR, and about the size of a fag packet. And actually in one; as the guy smoked it was a most innocuous enclosure.

        8. Man inna barrel

          Re: year 2000

          > ... single short beep with several-minute intervals to indicate a low battery, and due to the pitch and volume of that beep it's hard to pinpoint.

          A pure sine wave, or "beep", is very difficult to localise. The sound bounces around the room, and as you move about, the apparent location changes. I have done exactly the same dance around a room to find the annoying beeper in a drawer somewhere.

          Most natural sounds contain a range of frequencies, not just one sine wave. This appears to make it easier to locate the sound, perhaps by combining the localisation of several frequencies. I think this might be the reason that reversing warning sounds on newer trucks use a noise-like ksh-ksh sound, instead of the old beep-beep. At least it isn't "Warning! This vehicle is reversing!", which gets pretty annoying after the second time you hear it. It is all very well to keep people safe from being run over, but one has to consider the side effects on innocent bystanders.

        9. Aussie Doc
          Pint

          Re: year 2000

          OMG!!! Memories.

          When I were a young'un I built one of those 'cricket' circuits.

          You know the ones - it chirps loudly and annoyingly but stops as soon as everything goes quiet eg when someone says "Shhh. Listen. Can you hear it?"

          Fun times.

          Never got called back to that school again.

      2. NotAtHomeToday
        FAIL

        Re: year 2000

        For several years I was supposed to carry the pager at ALL times (and if necessary go and find a phonebox, this being the days when the only "mobile" phones were welded into big expensive cars), but I would click the fiddly three-position switch to silent vibrate mode when discretion was required. One such time being an in-laws family funeral. At least I thought I had. I was sure I had. Earth to earth, dust to ... beeeep beeeep beeeep.

        1. Snapper

          Re: year 2000

          Luckily I had my phone set to silent as I carried my Mother-in-Law's coffin back down the aisle.

          I think the undertaker on the other side from me twigged though...

  7. Pete 2 Silver badge

    Response time

    A conversation I earwigged, not being involved personally. But due to the nature of the issue I could not help but overhear.,

    The on-call support person on a particular friday night was someone who "knew their rights" and worked to the letter of their job description.

    At the time THE PHONE rang, they had just sat down to a romantic dinner at a local establishment. This was a time when any sort of mobile phone was quite a novelty, before the era of smartphones and laptops. It was not recorded whether that "romance" involved anyone else, but I digress.

    It turned out that the support person knew that the conditions of being on call required them to respond within 45 minutes. So 40 minutes after the phone rang (and apparently rang several more times) they called to find out what was interrupting their carefully planned, to coincide with being on call, night out.

    The details of the emergency were not part of the post-mortem, but downtime of the main server was said to be in the order of £5-figures per hour. What was apparent was the difference in interpretation of what "respond to a call" meant.

    It was explained to the support person that responding meant presenting themselves in body and mind at the premises to perform whatever was necessary to resolve the problem. This explanation was made loudly and in words that left no room for misunderstanding. And that when the on call phone rang, they were pretty damn well expected to answer it instantly.

    Production issues of this nature were quite rare, so the depth of the support person's understanding was never tested.

    1. John Brown (no body) Silver badge

      Re: Response time

      Ah, the ambiguity of language. Specifying a 45 minute response also requires a definition of "response".

      1. Martin an gof Silver badge

        Re: Response time

        When I was the on-call engineer for a radio station in the days when they couldn't expect me to afford a mobile phone on the salary they paid it was quite clearly stipulated I had to be within 10 minutes of a phone at all times and 30 minutes of the studios. It made simple things like walking the dog quite an excercise in planning a route between known-working phoneboxes.

        M.

  8. chivo243 Silver badge
    Windows

    I could only wish

    "She doesn't ask much about my work, though," he added wryly. The Missus is always askin' questions like What's a hash, why is it salted... What is a BSSID do...

    1. Anonymous Coward
      Anonymous Coward

      Re: I could only wish

      Well, hash is corned beef. Which can have salt added to it for extra flavor. Put it on crackers for additional zing.

      But don't post anywhere on the web that you want to sell "salted hash crackers"...

      (Credit: hovertext on http://thecodelesscode.com/case/207)

      1. Anonymous South African Coward Bronze badge

        Re: I could only wish

        Any idea why http://thecodelesscode.com has gone dormant?

        1. Dr. Ellen
          Windows

          Re: I could only wish

          I tried http://thecodelesscode.com just now, and it worked. I'm in the US, so my results may not be yours. Or, maybe, somebody was On Call and fixed it?

          1. Shooter

            Re: I could only wish

            I believe he may be referring to the fact that thecodelesscode hasn't been updated for over five years.

            I have no answer for that one.

  9. This is not a drill
    Pint

    Why is everything I type Blurry?

    Not out of hours, but definitely out of the office.

    I was one of the 2 senior Techs at a major retailer with operations across Europe. Myself, my collegue and our Boss had been invited to an Friday afternoon jolly organised by our telco which being an Australian company meant a beach party, with lots of beer, BBQ and beach volley ball, in central London.

    About 5.30pm we got a call from our Swiss operations, saying that their Internet provider had gone down and wouldn't be restored for some time, could we help. We mentioned to our Boss that we have considered that and had a contingency plan to route all their Internet traffic through the central core but we had never tested it. Luckily we had a office around the corner so we popped it to get onto our systems. The contingency plan involved making several changes to the central core routers and main HQ firewalls.

    This is where it gets interesting, we had been vigorously enjoying the hospitality of out Telco all afternoon, and were very "tired and emotional" to use political speak. As I typed the commands on the console, I had to get my colleague to read out everything I was typing as I was having issues focusing. Just before I hit 'commit' I turned to my Boss (who was also p****d) and said are you sure you want to do this, it could screw a lot of things up if we get it wrong? Yep. Commit. No alerts went off, good, phone the Swiss, they are now happy because everything was working again, good.

    And our Boss turned to us and just said "Thanks guys, let's get back to the party and never mention this to anyone".

    1. Xalran

      Re: Why is everything I type Blurry?

      > And our Boss turned to us and just said "Thanks guys, let's get back to the party and never mention this to anyone".

      I hope you managed to hang to that particular Boss for a long time, they are of a rare breed.

    2. This post has been deleted by its author

      1. This is not a drill

        Re: Why is everything I type Blurry?

        As I said Retail. Retail doesn't work 9.00-5.00pm. Across Europe some of our stores were open until midnight.

        It was quite a while ago and all involved including myself have since left that organisation.

    3. gotes

      A beach party in central London?

      Was that on the muddy shores of the Thames at low tide?

  10. mark4155
    Coat

    Regomiser requires an update!

    Firstly congratulations "Guy" on your 20 years of wedded bliss. Well done!

    Now "Guy" I think that our beloved Regomiser is in need of updating. The article starts with:

    "Today's story comes from "Guy" (not his name)"

    You see the obvious security holes in this? We know this is a man. So we can quickly rule out 49.6% of the population. More pedantically, out of 1,000 people, 504 are men (50.4%) and 496 are women (49.6%). [Source some BS off Google]. You see where this is going, the thin end of the wedge if you ask me!

    Getting to the point, whilst Regomiser has in the past been jolly helpful in disguising my identity, see my newly Regomised name below, I think we should get up with the kids and become PC (Not the ones with floppies and stuff) and program Regomiser to be gender neutral.

    Think "Les" "Chris" "Jo" - this list is endless.

    Do your best El Reg. Regomiser issued me with my new identity for which I am eternally grateful.

    Toodle Pip!

    Kram Renrut.

    1. Martin
      Happy

      Re: Regomiser requires an update!

      I'd have said "Guy" was gender-neutral. My daughter always says "Hey guys!" to all her friends - male and female.

      1. Dinanziame Silver badge
        Angel

        Re: Regomiser requires an update!

        It's gender-neutral until a man asks you if you would sleep with a guy...

        1. Anonymous Coward
          Anonymous Coward

          Re: Regomiser requires an update!

          But Guy is HOT! on fire.... especially on 5 of November.....

      2. rskurat
        Headmaster

        Re: Regomiser requires an update!

        and the truly woke call girls "dude"

    2. Already?

      Re: Regomiser requires an update!

      Jo is traditionally feminine; Joe is masculine.

      My son is a Joe, his mother aka my wife cannot bear seeing birthday and Christmas cards to him as 'Jo'.

      1. Anonymous Coward
        Anonymous Coward

        Re: Regomiser requires an update!

        "Jo is traditionally feminine; Joe is masculine."

        Depends on the culture. Some "obviously" female names in the West - turn out to be male eg Marion, Maria.

        Jo is also a common usage for Joseph eg Jo Johnson.

    3. Irony Deficient

      Today’s story comes from “Guy” (not his name)

      You see the obvious security holes in this? We know this is a man.

      Not necessarily; “his” could be epicene rather than masculine, and “Guy” could be short for “Guylaine”.

  11. DS999 Silver badge

    I didn't do much on-call

    But once I was a consultant responsible for storage on an SAP migration/upgrade (this was back in 2000 when people were dumb enough to think doing both at once to "save time" was a good idea) for a big US multinational you've no doubt heard of.

    It was scheduled to go live the day after Labor Day. I, of course, planned to drink heavily on or near the water hundreds of miles from home that weekend, as the storage had been stable for weeks and the remaining work was regarding replica copies of the DB for other purposes that weren't impacted by the go live. So while I brought along my laptop, I didn't think I would need it.

    As luck would have it, I got a call in the middle of the afternoon the day before Labor Day while out on a boat. Cell service being rather poor on that lake, I couldn't make out the voicemail too well but I figured there was no point in calling back because there was nothing I could do. It was not my boat, and I wasn't going to ask its owner to interrupt everyone else's fun to ferry me back. So a few hours (and few more voicemails) later when back on shore I checked my voicemail, and somehow all hell had broken loose and none of the production filesystems were mountable. Since the go live was scheduled for 6am the morning after next, this was kind of a big deal.

    So while everyone else was getting cleaned up to go to dinner and out on the town, I called into the conference line on my cell phone and dialed in to look at the problem using my friend's condo's landline. It took a few hours, but I eventually figured out what happened (the matching of BCV volumes had been undone by someone who kept a copy of an outdated storage script in their own directory and ran it, rather than using the official copies I maintained in a system directory) and was able to correct the issue, tell them what happened including fingering the guilty party (because I was pissed at having my plans disrupted) and waited on the call for everything to come back up and the SAP guys pronounce it working again. While doing that my drunken comrades returned from the evening's exploits and were like "you're STILL on the phone?" I'm sure everyone else on the call enjoyed the chaos in the background, which I deliberately did not mute again because I was pissed at having my plans disrupted by someone else's stupidity.

    Even though I was an hourly consultant so I was well paid for that, I did my best to avoid any contracts in the future that would have me commit to being "on call". I'd tell them I don't mind working on nights/weekends once in a while if something comes up but when I tell them I'm "away for the weekend" I will not be reachable!

  12. ShortLegs

    Sat in a meeting, phone rings: account manager with a seriously over-inflated sense of self-importance, demanding to know why I was not in his meeting. Explained I was in another meeting. Not good enough. cue a tirade to the effect I had 5 minutes to be there, or I'd be sacked. Unable to get a word in, handed phone to my boss... who informed him we were in New York, and could not make a meeting in the UK.

    On the subject of work relationships... a few years before I met my future girlfriend doing a weekend Netware server upgrade. She worked in the mid-range team and unexpectedly travelled halfway across England to see what we did. Oddly enough, most people thought we were married , due to having the same surname, and an incident the previous month with a software rollout. But thats another story.

    1. Robert Carnegie Silver badge

      Go on...

      I for one will be happy to hear another story? ;-)

    2. Swarthy

      Another Story

      Please share! :)

  13. Tom 7

    Always keep a 1/4 bottle of whisky handy when not actially being paid for on call stuff.

    'Hello the server whatever has gone tits up' 'Hang on a mo' glug glug glug 'Sorry been partying, there is no way I can help even if you are my best mate'.

  14. herman
    Flame

    Heavy breathing call

    I once got a support call when I was half way up the hill to a castle in Slovakia. The bladdy things are all on a steep hilltop.

    1. A.P. Veening Silver badge

      Re: Heavy breathing call

      The bladdy things are all on a steep hilltop.

      And not only in Slovakia, but world wide wherever there are steep hilltops available.

  15. Anonymous Coward
    Anonymous Coward

    8pm in the evening just as the IT ops is shuting up shop and going home, call from them "ups not working" - (I'm not on call, no such arrangement was in place) ok, "I'm away from home - be at least an hour and a half before I get there, will you be there? I have no key" ..... errrr ... and it was "my" fault of course.

  16. John Brown (no body) Silver badge

    On call and out partying

    There seems to be a lot of people who are officially on call, and yet seem to be having a completely normal social life, like going out and getting pissed. I've not been on call very often, and not for many years, but it seems highly unprofessional to be on call and then not being in a fit state to actually do the job. I can only assume those people worked for crappy companies who didn't pay extra for being on call. In my case, we were paid an on call rate for the time being on call as well as overtime rates for any travel/work done. Those who were on an on call rota didn't get the on call rate but instead got a higher basic salary because that was part of their normal job, but still got the overtime rates for travel/work done and had a chart showing times/length of call out and how that related to whether or when they had to show up at work the next day.

    1. NeilPost Silver badge

      Re: On call and out partying

      They pay extra for being on call but it’s the usual minimum they can get away with ‘allowance’, not proper wages for doing ‘regular work’.

      For arguments sake - at the bottom of the food chain which many of these tales are about - say it’s £5,200 annual allowance for this today.

      £100 ‘allowance’/week. If you loose half a Saturday

      £100/12 hours = £8.33/hour.

      As it’s not regular pay or overtime it’s also ‘not pensionable’ either.

      … and that’s also hoping you have no other calls outs on probably a 1 in 4 weeks ‘on call’ diluting the ‘pay’…. are you remotely surprised about people living their life whilst on call.

  17. Anonymous Coward
    Anonymous Coward

    I was considered a bit of a Luddite because I refused to have a company mobile phone when pagers became unfashionable. The advantage of the non-acking pager was you could do a Nelson if you received a message from PHB that deserved to be ignored.

    There was one amusing incident when a colleague had arranged to pick me up from a station coffee shop one morning - en route to a customer. Near the time when he was expected - my pager said "Stuck on M25 - 30 minutes delay". I had taken the first sip of a fresh coffee when the colleague rolled up. He hadn't sent the message - so someone somewhere was waiting in vain for the person who had sent it.

  18. Anonymous Coward
    Anonymous Coward

    The office had a dress code that boiled down to "a tie is not necessary on Friday". They were warned that if they called me in for an "emergency" when I was on holiday at home then I would arrive within 20 minutes - wearing whatever. Usually short shorts and a T-shirt. They never rang when I was in the suntrap though.

  19. This post has been deleted by its author

  20. NeilPost Silver badge

    5C

    “ "Opening the door to the machine room I was hit by a wall of hot air," Guy told us. "The thermometer just inside the door was reading 40°C, it should have been at 5°C."”

    Jesus …. No wonder the A/C broke, or is a typo and it should be 15C.

    IT equipment and Sun servers were in there, not chicken, milk and cheese.

  21. Nick London

    Nowt to do with IT, but taking spouse on work visit

    We are now empty nesters and a good client is a chain of leisure centres. They were doing remedial work which I had to inspect, so I did this on a Saturday and offered to take my wife for a drive in the Kent countryside and buy here lunch in a nice market down nearby.

    So the Mrs sat in the car whilst I inspected the remedial paving work to my specification. As I finished she came and wanted the loo.

    So I asked nicely and they let her use the changing rooms, this worried me because if she wanted to join it would be expensive to join. I was on tenterhooks.

    Anyway when she came out she said "you would have to pay me to join" so I breathe a sigh of relief. The sight of all those gym ladies in crop tops and clingy shorts put her off.

  22. Annihilator

    Guards or not, how long ago was this that temperature alerting wasn't a thing? I'm assuming this was in Telehouse. My run-of-the-mill NAS will email me when it gets a bit toasty...

  23. rskurat
    FAIL

    not even halfway through . . .

    . . . the comments and I've counted at least three David Brents

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like