Poor communication led to complete lack of communication Greetings and salutations, mighty reader, and welcome once again to Who, Me? in which Register readers like you share tales of the times your effort to deliver tech support went awry. This week meet a reader we'll Regomize as "Toby" who once worked as a marketing consultant and tech guy – quite the combination – for a digital …
The fact that this whole thing was "a day's work" probably tells you all you need to know about how well it was implemented.
I've been involved in a lot of integration projects of varying complexity, and a day wouldn't be long enough to write down the basic requirements for agreement, let alone to develop a working (and robust) end-to-end solution.
Ugh, I worked for a company that shall remain nameless but a huge amount of work was generated by a buggy app deployed to thousands of locations to integrate data and sync to the mothership. For years everyone lived with the problems, as they did get data, but the idiosyncrasies included not being able to the difference between a new record and an update to an existing record.
The guy who wrote it was still there and always bragged how he’d made in about 3 days. I constantly and successfully bit my tongue and did not point out that maybe that wasn’t something to be proud of, and maybe if he’d spent a bit longer, or gone back and updated the fucker, the company would have saved hundreds of thousands of pounds on labor alone, building system after system to handle the edge cases once the data landed at the mothership.
Classic “if I say complicated things everyone thinks I’m clever” guy…
"marketing consultant and tech guy"
The fateful combination of two providers for CRM and Web services seems a Hermetic mirror of Toby's two roles.
I had a similar issue with email in the days before on-site email servers and broadband. We used an ISDN connection to an external service which suddenly slowed down*. It turned out one of our users had put his Out Of Office on in whichever email client he was using without checking the "only send once" box. He'd then sent an email to someone external who'd done the same thing - result was multiple OOF messages bouncing between the two! Stopping the service, clearing the queue and fixing the OOF settings cured the problem.
*Users expected email to be instant, which it wasn't and isn't now - I had one senior manager complain that her colleague in the States hadn't received an email 2 minutes after she'd sent it. Being ISDN our email only connected every 15 minutes but I showed here there was nothing in the queue at our end and there was nothing we could do.
had similar back in the day. circa 2000, Netware 4.11 and Groupwise 4.x a couple of boxes running the Lab's IT, griefwise and NDS running on the same tin. A boffin left the lab to move to a Uni in the uk and managed to set up a out of office I've left email the lab to his uni account and then on his uni account a forward to his old account at our lab. Result, utter shit show mail loop! Took days to sort and as griefwise was on the same tin as NDS the snails pace o the box meant users couldn't login!
Fwd: Fwd: Fwd: Fwd: Fwd: Fwd: Fwd: Fwd: Fwd: Fwd: etc.
A colleague once triggered a mail flood between his mailbox and mine.
When me or the other senior tech on my team went on vacation we'd set up a regular OoO and, for a select number of addresses, a supplemental forwarding rule to the other senior to make sure requests from higher ups were properly followed up
Well, I went on holidays and set the rules, but my colleague had to take a day off and did the usual. Unfortunately the forwarding rules weren't set to just forward mail from those contacts, but also when they were CC'd... so when the first email form someone on the list was sent to his address (or mine, really don't remember who first triggered the glitch), mails started being forwarded from one account to the other until it froze the Exchange server)
I think he got a bit of an earful the next day, but nothing of consequence (except learning not to do it again, which is *a good thing*)
External listservs, internal groups, and out of office reply becomes mail bomb! A place I worked at had a Guy’s Night Out listserv, to plan outings sans SOs (of course everyone used their work address!). Most outings were breweries etc, a lot of the same guys belonged to the same internal mailgroups. Come holiday time and a poorly configured mail server, the out of office replies engaged, next stop 30,000 messages in the in boxes of every person in the GNO listserv, I had way more being in IT Support. It shouldn’t have been that bad, but some idiots thought asking wtf is going on with the mail server while replying all didn’t help matters.
My coat, as I’m done with mail servers!
We had a sales guy set his email to forward to his personal pipex account.
His next in line staff member went on holiday, and forwarded her email to him.
In comes a virus infected email our system didn't detect, but Pipex did. We had a single channel of ISDN, I got to see the tennis of rejected bounce -> forward -> rejected bounce as the email got slightly larger with the headers each time.
And Pipex's connection and sevrers were going to win regardless ;)
We had, for the time a relatively robust and successful CRM that used an on-site pop3/SMTP system on each of the physical hosts at each site to replicate to each other. The central CRM would reconcile the changes from different offices and send the replicated changes. This worked well for a long time.
We also had staff emails for each region on the pop3/SMTP system and they'd use Outlook to pull down the emails to the local workstations. We had Exchange at the head office but didn't have enough licenses for the rest of the offices.
We were coming up to the Xmas holidays and some of the regional managers wanted an out of office to trigger but didn't want to rely on having the Outlook running in order for the rules to trigger. The pop3/SMTP system had the feature that you could trigger an OOO response and you could set it at the user level or system wide. Being the holidays, I set it at the system level and for reasons I can't think why, I also set it at the head office. I think there were some non-exchange accounts for some manufacturing systems we used.
The next day I get an alert to say disks are full at head office and some of the regions. The CRM had been sending acknowledgement emails back and forth and send re-requests because the OOO email wasn't in an expected format etc. and I had to clear out hundreds and hundreds of thousands of garbage emails. I thankfully had remote access to all the systems and nothing fell over. The CRM ended up not even being out of sync but that took a while to verify.
Thankfully, I got companywide Exchange in after that and made sure to never mess with autorespond on the pop3/system in the future.
Many years ago, one of the engineers at a place I was working (in the UK) was assigned to one of the offices in the USA for a month. He was given an email account for that office so set a 3O on his UK a/c, and an autoforward to the USA one. Just before he returned, he set a 3O and autoforward on his USA a/c back to his UK one. Before he got back, his latter action had brought the whole network down!
This sounds like a change made without a spec at all. The fact that the local end got one day of work before the external provider started work suggests that someone came up with the idea and expected that everyone involved could begin work right now. I've worked for plenty of people who prefer to skip the planning stages because they don't understand why I would need any time. After all, they've just told me what they want, so why should I waste time planning when they've already done the planning? It often takes a lot of explaining to get them to understand that their idea is not a complete plan for its implementation, or more likely just that I'm not going to deliver what they want until I've done whatever it is I'm going to do.
> "Since we all know the cliché about what happens when you assume, we will not repeat it here"
This post has been deleted by its author
So why wasn't the infinite ping-pong of updates not noticed during testing?
I fell into the same trap when I was developing proof of concept code for a mesh network where packets were broadcast to systems which broadcast those onwards and I soon experienced the network flood I had neglected to consider. But it was immediately obvious what I had unwittingly done, what I had overlooked.
I got the impression that both parties had tested independently ..... but while the other party did not have their solution in place.
Then each party enabled their solution independently ..... leading to a loop the first time either was triggered.
A long time ago, a colleague would work to debug a program running for half the night on a super-computer (Cray brand).
Of course there was a very long loop, millions of occurrence ...
But very rarely on one model, a condition was happening, making the numbers wrong.
So, of course, the colleague put the following "if condition then send an email to me" ...
Yes, by now, everyone guessed it: the model he had run it with was now creating the condition EVERY SINGLE ITERATION.
The sysadmin in charge came to our office asking to the dude "hmmmm, what are you doing exactly with sendmail ?".
It finally took him half the day to clean-up the spool space, during which we had no new email ....
You get some fun stuff with Actuarial modelling, if you stuff up something basic enough. Errors that happen for every month in your projection, for every policy - my record is a log file with 4.8 million error messages in it. Took twenty minutes to open the file with the tools I had at the time.
Many years ago, I worked for a contractor at a US civil agency, supporting mostly WordPerfect Office. The email did not have auto-forward, so our programmer wrote one. He wrote it in COBOL, since that's what he knew best, and I don't recall how he kicked it off--a batch job, probably.
it had no loop detection. We fielded many calls from users upset to find that their in-boxes had thousands of repeated messages. It seems to me that I tried to introduce loop detection, though given that I knew, and know, essentially no COBOL, I can't think how.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2025
