Reply to post: Re: exim

Re: exim

@W.S.Gosset

"Your mailserver can get jammed before you have time to react..."

We had a similar experience three to four years ago - completely self-inflicted.

After having used Spiceworks for a couple of years, mainly to track a large client's IT inventory and using it internally to handle tickets, the admin who looked after Spiceworks decided it was time to implement an e-mail ticketing system that the client could use to raise tickets (we used to open tickets as faulty equipment came in, but the burden on us was becoming increasingly more onerous).

So, one particularly slow Friday, late in the year when everyone was in relaxed mode, he set up the rules and we did a couple of tests. After ironing out some minor niggles, we were ready and the system went live.

The next step was to send out a company-wide (to the client) mail, from Spiceworks, advising them of the improvement to service, as well as a short run-down/instruction manual on how to log calls/raise tickets.

About ten minutes later we received the first call about e-mail not going through, as well as people having problems accessing mail.

A short discussion determined it had to have something to do with implementing the ticketing system, but what? The feeling of dread started sinking when we discovered that we could not log on to either the mail server or to Spiceworks. The sessions we had open had also become unresponsive.

We were able to log on to both servers via ssh (albeit extremely sluggish), only to find that the mail queue was brimmingly full and new mails being added in copious amounts.

I was given the task of clearing the mail queues as fast as possible, whilst he struggled with trying to stop Spiceworks sending mail. Two or three hours later we managed to wrest control back of both servers and then stopped the mail server (the end of the day was imminent in any case, and nobody could send mail in any case).

We still had no clue as to what had happened, but we were reasonably certain it had to have been a rogue rule or two that caused the mayhem. The first clue came when I noticed that most of the messages had been between Spiceworks and a very small number of users. Inspecting two or three of those mailboxes, brought to light that they were on leave and had set up out of office messages, as per policy, advising clients of their absence and who to contact instead.

When I mentioned that to my colleague (at the time I was not privy to all the rules he had set up in Spiceworks), the awfaul truth dawned; he had a rule that Spiceworks would send an auto-response to the person who had raised a ticket, acknowledging receipt of same, et cetera.

So when Spiceworks received the out of office message, it would send out its acknowledgement, generating an o-o-o message, which... You get the message.

Handling that particular scenario was not easy nor straightforward, but by the next Tuesday we had it sorted.

Anon for obvious reasons.