8x8 Work Login failures and 8x8 Work applications performance issues

Incident

8x8 Work Login failures and 8x8 Work applications performance issues

Status: closed
Start:
End:
Duration: 2 hours 33 minutes
Affected Components:
Americas EMEA APAC Work Calling SSO SSO SSO Endpoints WorkD (VOD)
Investigating

8x8 is investigating reports of login failures to 8x8 Web services via login.8x8.com. Chat and presence in the 8x8 work applications have also been reported to not be properly functioning or loading. We'll provide updates as we confirm and validate the reports. 

Investigating

8x8 has confirmed impact to 8x8 Work logins to web and application logins as well as some performance issues with 8x8 Work chat, presence and overall functionality. We are currently investigating and will provide updates as they become available. 

Update

Our engineers are currently working to resolve the issue as a matter of urgency. We will continue to provide updates as they become available.  

Investigating

8x8 engineers continue to work towards the resolution of the issue. We will continue to provide updates as they become available and we apologize for the inconvenience. 

Update

We are doing our very best to get this issue resolved as a matter of priority, we thank you for your patience. We will provide further updates as they become available and we again apologize for the inconvenience.

Update

8x8 continues to work towards restoring new logins to 8x8 work and authentication services. All required teams are currently engaged and investigating the cause. Updates will continue to be given when significant updates become available. We apologize the delay in resolving this issue.

Update

Our engineers are currently working to resolve the issue as a matter of urgency. We will continue to provide updates as they become available. 

Identified

8x8 has identified the cause and has performed some mitigation that has improved login success. We're continuing to confirm and monitor and will provide an update once we confirm the mitigation has been successful. 

Monitoring

8x8 has confirmed that logins have been successful since mitigation has been applied. We currently expect all symptoms to be resolved. We are continuing to monitor for stability and will provide further updates once we confirm full resolution to close out this alert. 

Resolved

8x8 has monitored and confirmed that its mitigation has resolved the issues with logins and 8x8 Work desktop and mobile performance. We are performing a full root cause analysis and we apologize for the interruption this has caused.