Posts by malloctheballots

Rackspace rocked by ‘security incident’ that has taken out hosted Exchange services

Re: Business Continuity

"I'd be willing to best most(90%+) of the customers did not take their own backups, just like most likely most office 365 customers don't take their own backups."

Point well taken but the marketing information and the past history of RAC was that they did it all and risk was much much lower.

Also exchange went into something called cached exchange mode by default and you had to turn it off to get all your mail all the time. Otherwise it constantly compensates for data set sprawl by optimizing your folders - leaving things on the server until you click on 'see mail on server' every time you start the program or leave the Inbox. Even your searches could hide real mail because of MS's crappy databased system.

There USED to be a PST file that held everything locally and that allowed for easy reconstruction of your data if the server died. But today you only get a fraction of what is available unless you turn that default off and wait patiently for 2-5 hours to pull all your mail down.

Re: Time to Change Providers

You would be better off by using a service like GoDaddy as a go-between or just biting the bullet and going to MS direct. We are an MSP on the East Coast of the US and offer to move people over and place them on 365 hosted.

Either way, it is wise to get your tenant elsewhere.

Re: Business Continuity

Nate - I appreciate your background and loving the background in Linux.

I will tell you as an MSP and on prep administrator for Exchange and as a partner in Rackspace, while your points should be considered by anyone arguing the point, RAC also has broken their SLA on many levels. They under represent the issue, and they are no longer the company they once were. It should hav never happened because it never happened for over 15 years. These things don't just happen based on clever idiots renting BotNets who are clueless. They happen due to negligence. No one five years ago would agree that RAC would have a breach. As subscribers we didn't pay attention to the writing on the wall. If you look at their past white papers and work performed to create a secure environment they had the elements in place. Unfortunately they have abandoned the purpose of their business and that rotted the business entirely. It only worked with the priority of security of customer data, and that includes hiring the best, keeping the best and honoring their workforce.

If you had been a part of their 'pay for' experience, 15 or 10 years ago, you would find an immediate support call answer, and relevant support help. If something needed to be escalated, it would reach a solution rapidly. It ran as as business, as you would expect. Over the past five years, before COVID, things had started to slid backwards. I suspect it was because the company has reinvented itself in ownership and in C level management over time.

This is a GRID of machines. Backups and HA security was throughout the company housed data. They had insane expertise for all things Exchange, which has been upgraded over the years from 2000, to 2003, 2007, 2010, 2013, 2016... Rackspace never upgraded based on release dates, but instead based on awaiting Dev testing and re-testing in stability. Their mission was providing the best most stable and most secure environment for business users at one point. The reason they held on to older versions of exchange was because they were stable. Also, they grew as a business to understand cloud dynamics and hired experts who understood security and security concerns within cloud. They were great.

But something happened, and I suspect it has to do with both management and engineering. They changed their management and this is when things collapsed. Internally I believe their mission changed based on ignoring the past success of the business culture in engineering and the entire company. They hired management with different goals and priorities. I suspect at the root of the matter is based around unqualified employees in upper management who should have never been promoted nor hired.

Instead of actively going after phishing attempts daily, and seeking those recent endless phishing attempts to access their billing system in SPAM, they got caught. Their systems of security had been compromised from the edge all the way to employee training (with lots of neglected protection point opportunities in-between).

There is no more stark example of what happens when you cater to the whims of both the wrong VC Capital group "best advice: (DROP YOUR COSTS)", a true lack of ethic internally to reward employees who are qualified (eg raises WHEN WARRANTED, training opportunities for employee knowledge growth, recognition for successes that are meaningful to the corporate mission of fanatical support, pushing for recognizing valuable women in STEM with continued learnings and recognizing their engineering growth paths, and giving appreciation to the people that endlessly drive the product engineering and security). When you hire people who can't leave the social media mirror (narcissism) and ignore the machines around them, you compound the problem.

It's happening in all social media companies and in many tech companies.. it is a FAIL. The importance of an employee's 'feelings' is not in line with the fiscal responsibility of a board at a public company. Hiring careerist HR specialists who only know how to divide workforce based on how 'unfair' realities will destroy companies.