Rackspace hosted Exchange suffered a catastrophic interruption starting December 2, 2022 and is still continuous since 12:37 AM December 4th. At first referred to as connectivity and login concerns, the guidance was ultimately updated to announce that they were dealing with a security event.
Rackspace Hosted Exchange Issues
The Rackspace system decreased in the morning hours of December 2, 2022. At first there was no word from Rackspace about what the issue was, much less an ETA of when it would be solved.
Consumers on Buy Twitter Verification reported that Rackspace was not reacting to support emails.
This has actually been rather the day with #Rackspace. Every hosted exchange client has actually been down for 14 hours approximately. Assistance isn’t reading/responding to tickets. Updates are unhelpful.
I am concerned now that they succumbed to something bad like the ProxyNotShell PoC hack. https://t.co/jchKsAO3Z7
— Joe Sinkwitz (@CygnusSEO) December 2, 2022
A Rackspace consumer privately messaged me over social networks on Friday to relate their experience:
“All hosted Exchange customers down over the past 16 hours.
Uncertain the number of companies that is, however it’s significant.
They’re serving a 554 long hold-up bounce so people emailing in aren’t familiar with the bounce for several hours.”
The main Rackspace status page provided a running upgrade of the blackout however the preliminary posts had no details aside from there was a failure and it was being examined.
The first authorities upgrade was on December second at 2:49 AM:
“We are investigating a concern that is impacting our Hosted Exchange environments. More details will be posted as they appear.”
Thirteen minutes later Rackspace began calling it a “connection concern.”
“We are examining reports of connection concerns to our Exchange environments.
Users might experience an error upon accessing the Outlook Web App (Webmail) and syncing their email client(s).”
By 6:36 AM the Rackspace updates described the continuous issue as “connectivity and login issues” then later that afternoon at 1:54 PM Rackspace announced they were still in the “investigation phase” of the outage, still attempting to find out what went wrong.
And they were still calling it “connection and login concerns” in their Cloud Office environments at 4:51 PM that afternoon.
Rackspace Recommends Moving to Microsoft 365
Four hours later Rackspace referred to the situation as a “significant failure”and began providing their clients totally free Microsoft Exchange Plan 1 licenses on Microsoft 365 as a workaround until they comprehended the issue and might bring the system back online.
The official guidance mentioned:
“We experienced a significant failure in our Hosted Exchange environment. We proactively closed down the environment to avoid any further concerns while we continue work to restore service. As we continue to resolve the origin of the problem, we have an alternate solution that will re-activate your capability to send and get e-mails.
At no charge to you, we will be supplying you access to Microsoft Exchange Plan 1 licenses on Microsoft 365 up until more notice.”
Rackspace Hosted Exchange Security Occurrence
It was not till nearly 24 hours later at 1:57 AM on December 3rd that Rackspace formally announced that their hosted Exchange service was suffering from a security event.
The statement further exposed that the Rackspace service technicians had actually powered down and detached the Exchange environment.
“After more analysis, we have figured out that this is a security incident.
The known effect is isolated to a part of our Hosted Exchange platform. We are taking essential actions to examine and secure our environments.”
Twelve hours later on that afternoon they updated the status page with more info that their security team and outdoors professionals were still dealing with resolving the blackout.
Was Rackspace Service Impacted by a Vulnerability?
Rackspace has not released information of the security event.
A security occasion typically includes a vulnerability and there are 2 severe vulnerabilities currently in the wile that were patched in November 2022.
These are the 2 most existing vulnerabilities:
Microsoft Exchange Server Server-Side Request Forgery (SSRF) Vulnerability
A Server Side Request Forgery (SSRF) attack allows a hacker to check out and alter data on the server.
Microsoft Exchange Server Remote Code Execution Vulnerability
A Remote Code Execution Vulnerability is one in which an opponent is able to run malicious code on a server.
An advisory released in October 2022 explained the effect of the vulnerabilities:
“An authenticated remote enemy can carry out SSRF attacks to escalate privileges and execute arbtirary PowerShell code on susceptible Microsoft Exchange servers.
As the attack is targeted against Microsoft Exchange Mailbox server, the aggressor can potentially access to other resources by means of lateral movement into Exchange and Active Directory site environments.”
The Rackspace interruption updates have actually not suggested what the particular problem was, only that it was a security event.
The most present status upgrade as of December 4th stated that the service is still down and customers are encouraged to move to the Microsoft 365 service.
Rackspace published the following on December 4, 2022 at 12:37 AM:
“We continue to make development in dealing with the event. The availability of your service and security of your information is of high significance.
We have actually dedicated extensive internal resources and engaged first-rate external expertise in our efforts to reduce negative impacts to customers.”
It’s possible that the above kept in mind vulnerabilities relate to the security occurrence impacting the Rackspace Hosted Exchange service.
There has actually been no statement of whether customer info has been compromised. This occasion is still continuous.
Featured image by Best SMM Panel/Orn Rin