skip to Main Content

This early AM on the East Coast, Teams experienced an access outage. The Exoprise sensors detected this outage an hour before Microsoft published a report on the issue. Here’s an example of what you get when you attempt to sign in, fresh, to Microsoft Teams:

Teams Outage April 27th, 2021
Teams Outage April 27th, 2021, Error 401

Early Detection

And here’s what early detection looks like. Our Teams sensors are always testing Teams 24/7 and in this case the sensors started noticing issues at 2021-04-27 09:57:08 UTC. They accumulated errors during the testing period for up to 5 minutes and then notified us across the environment. Here’s an example email from one of the sensors running on our public sites:

Early Teams Outage Detection, April 27th, 2021
Early Teams Outage Detection, April 27th, 2021, HTTP 401 Errors

Sample Dashboard

Here’s what our dashboard looked like, lit up this AM

Teams Outage Dashboard
Teams Outage Dashboard, April 27th

Microsoft Status Update

Since Exoprise has integrated Microsoft Service Communications Feeds (and dissemination), we see that Microsoft first published an investigatory note at 6:45 AM ET (10:45 AM UTC). That’s just about an hour after Exoprise Teams Monitoring detected issues.

Can’t access Teams

TM252802, Microsoft Teams, Last updated: April 27, 2021 6:50 AM
Estimated start time: April 27, 2021 6:24 AM

Issue type: Incident

Status: Service degradation
Manage notifications for this issue

User impact:
Users may be unable to access Microsoft Teams.

Latest message: April 27, 2021 6:47 AM
Title: Can’t access Teams
User Impact: Users may be unable to access Microsoft Teams.
More info: Users able to access Microsoft Teams may experience degraded performance with multiple features.
Current status: We’re analyzing system telemetry to isolate the source for the issue. In parallel, we’re reviewing recent changes made to the service as a potential root cause.
Scope of impact: Initial indications suggest that any user could be affected by this issue.

Even though Microsoft indicates that the estimate start time was 6:24 AM they didn’t publish a notice for another twenty minutes. And the actual outage started 20 minutes earlier according to our monitoring.

Ongoing Updates

PublishedTime:Tue, 27 Apr 2021 12:14:04 +0000
MessageText:Title: Can’t access Teams User Impact: Users may be unable to access Microsoft Teams through both the web and desktop client. More info: Users able to access the Microsoft Teams may experience degraded performance with multiple features. These include but are not limited to: – Can’t send or receive messages – Can’t join meetings – Some channels may not appear – Can’t join channels – Can’t join chats Current status: We’ve identified the root cause as a recent configuration change. We’ve reverted the change and are seeing signs of recovery. Users still experiencing impact are encouraged to restart their client to expedite recovery. Scope of impact: Any user could be affected by this issue. Next update by: Tuesday, April 27, 2021, at 2:00 PM UTC
PublishedTime:Tue, 27 Apr 2021 11:39:34 +0000
MessageText:Title: Degraded chat functionality User Impact: Users may experience degraded chat functionality. More info: Impacted functionality includes, but is not limited to: – Can’t send or receive messages – Some Channels may not appear – Can’t join channels – Can’t join chats Current status: We’ve identified errors within the chat service and are investigating these further to identify the source of the issue. Scope of impact: Initial indications suggest that any user could be affected by this issue. Next update by: Tuesday, April 27, 2021, at 1:00 PM UTC
PublishedTime:Tue, 27 Apr 2021 10:47:42 +0000
MessageText:Title: Can’t access Teams User Impact: Users may be unable to access Microsoft Teams. More info: Users able to access Microsoft Teams may experience degraded performance with multiple features. Current status: We’re analyzing system telemetry to isolate the source for the issue. In parallel, we’re reviewing recent changes made to the service as a potential root cause. Scope of impact: Initial indications suggest that any user could be affected by this issue. Next update by: Tuesday, April 27, 2021, at 12:30 PM UTC

Preliminary Root Cause for TM252802

A recent configuration change resulted in specific feature settings to include an incorrect value, resulting in impact to the service. We’re still investigating the specific details, and we’ll provide more details in the Post Incident Report (PIR). Next steps: – We’re reviewing the details on the configuration file update to identify the source of the issue. We’ll publish a post-incident report within five business days.

Team Exoprise

Team Exoprise represents multiple people in the engineering, sales and marketing department here at Exoprise. It takes a village.

Back To Top