Multifactor authentication issue hitting North American Azure, Office 365 users
Windows 10
A widespread multifactor authentication (MFA) issue is hitting several Microsoft customers in North America this morning, October 18. The exact cause of the problem is not clear at the moment, but Microsoft's engineering team says it is working on it.
I've heard from some customers that they've been affected since at least 10am ET.
Microsoft's Azure status page currently (as of 11:15am ET) says:
"Customers in North America are experiencing issues with Sign-in when Multi-Factor Authentication is enabled. The engineering team is currently investigating the issue and will send out an update as soon as possible."
The Microsoft 365 Status twitter account, as of 10:45am ET, said:
"We're investigating issues where users may be unable to access the admin center when using MFA. We'll provide an update shortly."
I've asked Microsoft for an update. No word back so far.
Users are reporting they cannot sign in to Office 365 or access any of their Office 365 apps and services. Office 365 uses Azure Active Directory for authentication.
This is not the first time that a widespread MFA issue hit Azure and Office 365 customers. In November 2018, MFA issues hit a number of Azure and Office 365 customers in two separate incidents in two consecutive weeks.
Update (11:20am ET). The Azure status page now says:
"Starting at approximately 13:42 UTC 18 Oct 2019, customers in North America are experiencing issues with Sign-in when Multi-Factor Authentication is enabled. The engineering team is currently investigating the issue and will send out an update as soon as possible."
Update (11:30am ET). The @MSFT365Status account says Microsoft has confirmed: "multiple Microsoft 365 services are impacted by the MFA outage."
"We're reviewing system logs to identify the source of the problem. We'll continue to post updates via MO193431 and on http://status.office.com."
Update (11:40am ET). The status.office.com page now says
"User Impact: Users may be unable to sign in to Microsoft 365 services.
More info: Users may not receive authentication requests via phone call, SMS or within their authenticator app.
Current status: We're analyzing authentication logs to isolate the cause of the issue.
Scope of impact: This issue could affect any of your users if they utilize MFA to access Microsoft 365 services.
Next update by Friday, October 18, 2019, at 4:30 PM UTC"
Update (12:20pm ET): Microsoft's Azure account says the outage began at 9:30am ET and that the company is still looking into all kinds of possible causes.
"We are aware that customers in North America are experiencing issues with completing MFA challenges. This issue started at approximately 6:30 AM PST and is ongoing. We are aware of how critical this is to your business and security. We are actively investigating the issue as our absolute top priority and working for the fastest-possible resolution.
At this point, we are investigating potential root causes including but not limited to config changes, recent patches, and networking issues, but we have not isolated any root cause or mitigation yet."
Update (3pm ET): Microsoft says the issue was resolved around 12:50pm ET. A root-cause analysis will be available in the coming days, officials say
"Preliminary root cause:
Engineers are continuing to investigate the root cause. A follow-up RCA will be provided in the coming days.
Mitigation:
Engineers took corrective action to fully mitigate the incident. Further details on mitigation actions will be provided in the RCA.
Next steps:
Engineers will continue to investigate to establish the full root cause and prevent future occurrences."
Update (October 21): Microsoft has published some details on its preliminary root-cause analysis about last week's outage. The Azure Status history page notes:
"The trigger of the issue was severe packet loss between Microsoft and a third-party service. The packet loss occurred in a network external to Microsoft but on the route to the third-party service. The packet loss was greater in severity and duration than previously measured. The combination of severe packet loss and morning peak load in North America resulted in Azure MFA service degradation in North American data centers. Azure MFA in the rest of the world did not experience a degradation of service."
Officials say that "changes have been made to improve resilience and throttling to better protect against similar issues connecting to third-party services." Also, "we are accelerating work that was already in progress which is designed to protect against a total loss of connection to the third-party communications service."