Some of our customers experienced issues with our Rainbow services over the last few days. If you were affected, we sincerely apologize for the inconvenience it caused your organization. The entire Rainbow operations team worked non-stop to restore the impacted systems as quickly as possible. Services were successfully restored for all users on November 18th, at 3:45 pm CET.
We continue working around the clock to identify all the root causes that unfortunately coincided, causing the services outages we experienced. We will notify you as soon as the fix is implemented, which will prevent this from happening again. Following our continuous improvement process to reach the best quality and stability, we will extend our automated testing coverage to verify Rainbow resiliency against potential risks.
Let me share some background information with you:
Rainbow uses numerous data centers from various hosting providers which are interconnected across the globe. Our data center distribution policy secures the right level of performance and enables us to meet rigorous data privacy requirements in all countries. A combination of multiple coincidences caused the disruption. One of the identified problems was a network congestion in one of our main data centers, which impacted our core modules.
We are implementing a contingency plan to prevent this issue from reoccurring:
- [November, 30]: Further increasing the backend capacity beyond nominal traffic to prevent any congestion impact, if it were to happen again.
- [November, 30]: Fortifying our defense mechanisms to ensure our services continue running even if our hosting network encounters any issue.
- [December, 24]: Ensuring our information and follow-up system proactively delivers real-time notifications as well as offering a live dashboard of our entire Rainbow infrastructure.
We'd like to take this opportunity to apologize once more for the inconvenience caused by these incidents. Rest reassured that both the Rainbow and the ALE teams are working on overcoming all challenges that may arise with the greatest dedication and passion. We remain available to clarify any concern you may have.
We count on you to help us continue to deliver the most advanced collaboration solutions in the cloud with Rainbow.
EVP, Cloud Communication Business Division