Security is a top priority for Microsoft and our customers, and we continue to work very hard to keep customer data secure from cyber-threats, while ensuring compliance with evolving regulations. As part of our continued work to help you protect your Exchange Servers, in the September 2021 Cumulative Update (CU) we have added a new feature called the Microsoft Exchange Emergency Mitigation service. This new service is not a replacement for installing Exchange Server Security Updates (SUs), but it is the fastest and easiest way to mitigate the highest risks to Internet-connected, on-premises Exchange servers prior to installing applicable SUs.
Security Vigilance with on-premises Servers
Cyberattacks continue to increase in frequency and sophistication and as we have said repeatedly, it is critical to keep your on-premises infrastructure secure and up to date, including all your Exchange servers. This is a continuous process in which you:
- Take inventory Use the Exchange Server Health Checker script to see if you are behind on your on-premises Exchange Server updates.
- Install needed updates Use the Exchange Update Wizard to get directions and install the latest updates on your Exchange server(s).
Important With the September 2021 CU release, only the June 2021 and September 2021 CUs are supported for Exchange SUs. If you are on Exchange 2016 or Exchange 2019 and not yet running a CU that is supported for SUs, you should install a supported CU and applicable SUs now.
After the release of the March SUs, we learned that many of our customers weren’t ready to install them because they were not running a supported CU. Based on our customer engagements, we realized that there was a need for a simple, easy to use, automated solution that could help customers quickly protect their on-premises Exchange servers, especially those who did not have dedicated security or IT teams to apply critical updates.
To help our customers, we released the Exchange On-premises Mitigation Tool (EOMT). The EOMT is a one-click tool that applies interim mitigations to an Exchange server to proactively minimize vulnerable attack surfaces until the admin can install an available SU. This was our recommended approach for Exchange deployments with Internet access and for those who needed to quickly mitigate their risk while they prepared to update their servers.
In addition to releasing EOMT last March, we also released automatic mitigation for on-premises Exchange Server in Microsoft Defender Antivirus and System Center Endpoint Protection. As with EOMT, these were interim mitigations designed to help protect customers who needed extra time to install the available SU.
In the June CUs for Exchange 2016 and Exchange 2019, we introduced another safety measure for on-premises Exchange customers: integration between Exchange Server and the Windows Antimalware Scan Interface (AMSI). AMSI integration in Exchange Server provides the ability to scan content in HTTP requests and block a malicious request before it is handled by Exchange Server, defeating common attempts to exploit unpatched software and deploy malware.
Exchange Server Emergency Mitigation
Building on that approach, in the September 2021 CUs we are introducing a new component in Exchange Server called Emergency Mitigation (EM). Like other Exchange Server components, EM runs as a Windows service on Exchange Server. It is a built-in version of the EOMT that works with the cloud-based Office Config Service (OCS) to provide protection against security threats that have known mitigations. The OCS is the same online configuration service used by Office clients.
The use of EM is optional for customers who want Microsoft to create and automatically apply vulnerability mitigations to their Exchange servers. If your organization doesn’t want to use EM, an admin can disable it and continue to use the EOMT to manually mitigate threats.
Here’s how EM works. Once an hour, the EM service checks the OCS for mitigations by calling into https://officeclient.microsoft.com/getexchangemitigations. Since in the future mitigations may be released at any time, we chose to have the EM service check for mitigations hourly. A mitigation is an action or set of actions used to secure an Exchange server from a known threat. If Microsoft learns about a security threat and we create a mitigation for the issue, that mitigation can be sent directly to the Exchange server, which would automatically implement the pre-configured settings. The mitigation package is a signed XML file that contains configuration settings for mitigating a known security threat. Once received by the Exchange server, the EM service validates the signature to verify that the XML was not tampered with and has the proper issuer and subject, and after successful validation applies the mitigation(s).
As with the EOMT, the work performed by the EM service is intended as a temporary and interim mitigation for customers until they can apply an SU that fixes the vulnerability. As stated previously, the EM service is not a replacement for Exchange SUs, but it is the fastest and easiest way to mitigate the highest risks to Internet-connected, on-premises Exchange servers prior to updating.
Understanding Exchange Server Emergency Mitigation
The EM service is part of Exchange Server and will be installed automatically on Mailbox servers when you install the September 2021 (or later) CU. It will not be installed on Edge Transport servers. Once installed, EM is an optional service that can be disabled by an admin. In fact, on Exchange servers without Internet connectivity, you’ll want to disable EM because it can’t work without Internet connectivity. In those cases, or when you don’t want automatic mitigation, we recommend using the EOMT to apply available mitigations manually. If you have outbound Internet connectivity but are using restrictions, you will need to enable outbound connectivity to the OCS, which is https://officeclient.microsoft.com.
Emergency Mitigation Prerequisites
The addition of the EM service means additional pre-requisites for installing Exchange. The EM service requires the IIS URL Rewrite module v2 to be installed on the Exchange server. If this module is not installed on the server when you install the CU, you’ll receive an Error message during the Prerequisite Analysis phase of Setup. Regardless of whether you plan to use EM, the IIS URL Rewrite module is a pre-requisite for installing Exchange, starting with the September 2021 CU. When applying a mitigation through the IIS URL Rewrite module, EM may make changes to the web.config file on the Exchange server. These changes are done automatically, and either they all succeed, or web.config is rolled back to the customer’s original state.
If you are running Exchange Server 2016 on Windows Server 2012 R2, you’ll also need to first install the Update for Universal C Runtime in Windows (KB2999226). If this update is not installed, Exchange Setup will not be able to proceed.
Finally, the Exchange server will need connectivity to the OCS, specifically to https://officeclient.microsoft.com. Without this connectivity, the EM service can’t function.
Understanding Mitigations
A mitigation is an action or set of actions that are taken automatically to secure an Exchange server from a known threat that is being actively exploited in the wild. The EM service (like the EOMT) is able to apply multiple mitigations, including:
- Implementing an IIS rewrite rule to filter malicious HTTPS requests;
- Disabling an Exchange service; and
- Disabling a virtual directory or app pool.
Actions performed via a mitigation include URL rewriting, stopping/starting app pools and services, changing authentication settings, and modifying other configuration settings. This means that to protect your organization and mitigate risk, the EM service may automatically disable features or functionality on an Exchange server (just as the EOMT did in March).
If your organization has an alternate means of mitigating a known threat, you may choose to block any EM service mitigations for that threat from being automatically applied. Details on how to do that can be found later in this blog post.
Optionally Send Diagnostic Data to Microsoft
When installing the September 2021 (or a later) CU, you’ll also notice a change to the License Terms acceptance process. We have added the ability to send diagnostic data from your Exchange servers related to mitigations. This data is sent to the OCS when the EM service checks for available mitigations.
When using the GUI version of Setup, you’ll see a new License Agreement screen, as shown below:
You can choose one of the following:
- I accept the license agreement and will share diagnostic data with Microsoft: This is the default option which accepts the license agreement and enables sending data to Microsoft.
- I accept the license agreement, but I’m not ready to share diagnostic data with Microsoft: This option accepts the license agreement but disables sending data to Microsoft.
- I do not accept the license agreement: If you don’t accept the EULA, you cannot install the CU (as with all CUs).
These options are also available via an unattended command-line or scripted setup through the use of new Setup switches:
- /IAcceptExchangeServerLicenseTerms_DiagnosticDataON: Use this new Setup switch to accept the license terms and send optional data to Microsoft.
- /IAcceptExchangeServerLicenseTerms_DiagnosticDataOFF: Use this new Setup switch to accept the license terms and disable sending optional data to Microsoft.
Important The previous Setup switch used to accept the license agreement (/IAcceptExchangeServerLicenseTerms) has been removed from Exchange Setup and replaced with the above two new Setup switches.
After Setup has completed, an admin can enable or disable sending the optional data to the OCS on any Exchange server using Set-ExchangeServer. For example, use:
- Set-ExchangeServer -Identity <ServerName> -DataCollectionEnabled $false to disable sending optional data to Microsoft.
- Set-ExchangeServer -Identity <ServerName> -DataCollectionEnabled $true to enable sending optional data to Microsoft.
If sending optional data to Microsoft is enabled, each Exchange server sends the following information to the OCS when checking for mitigations:
- Exchange Server build: The Exchange server version (CU and SU build number information).
- Emergency Mitigation service state: Information about the admin-configured behavior of the EM service (for example, whether to send data and/or automatically mitigate).
- Immutable Device ID: Unique identification for the server.
- Immutable Org ID: Unique identification for each Exchange organization in your environment.
- Applied mitigations: A list of all mitigations that have been applied and their status.
- Blocked mitigations: A list of all mitigations that have been blocked by an admin.
To better keep Exchange Server secure and up to date and to identify and mitigate threats, we may update what data is collected in future releases. For the most current information on what data is collected when sending optional data is enabled, please see the product documentation.
Sample PING Mitigation
After the September 2021 CU is installed, the EM service will communicate with the OCS to check for mitigations. To ensure this process is working correctly, and to allow admins to work with and learn about this new feature, we will send a sample mitigation called PING to the EM service. This sample mitigation is used solely for verifying the health of the EM service/OCS pipeline end-to-end and to allow admins to interact with this new feature. It’s really just there to test everything in the real world before we release any actual mitigations.
The sample PING mitigation contains no configuration settings, and it does not take any actions on or make any changes to the Exchange server. But without this initial sample mitigation, admins would not be able to learn how to manage mitigations before any real ones are released.
Because the sample mitigation does not make any changes, there’s no way to remove it. However, using the processes described later in this blog, admins can block the sample mitigation or choose to keep it applied.
Controlling automatic mitigation in your environment
Admins have visibility and control over mitigation behavior using PowerShell. For example, you can find details on applied mitigations and their status using scripts included with EM, and you can find details of mitigations blocked by an admin using PowerShell and in the Windows Event Log.
Viewing Mitigations
A detailed list of available mitigations can be viewed using the Get-Mitigations.ps1 script, as shown below.
Enabling and Disabling Mitigations
All applicable mitigations are enabled by default. An admin can enable and disable mitigations at an organizational level or at the Exchange server level. For example:
- Set-OrganizationConfig -MitigationsEnabled $false is used to disable automatic mitigation org-wide. By default, this is set to True. When set to False, the EM service will check for mitigations hourly but won’t automatically apply mitigations to any Exchange server in the organization, regardless of the value of MitigationsEnabled at the server level.
- Set-ExchangeServer -Identity <ServerName> -MitigationsEnabled $false is used to disable automatic mitigation on a specific server. By default, this is set to True. When set to False, the EM service checks for mitigations hourly but won’t automatically apply them to the specified server.
The combination of these two settings determines the behavior of the EM service on each Exchange server in the organization, as detailed in the following tables:
|
Org Setting |
True |
EM will automatically apply mitigations to the Exchange server when both are True. |
|
Server Setting |
True |
|
|
|
||
|
Org Setting |
True |
EM will not automatically apply mitigations to a specific Exchange server when the Org setting is True, and the Server setting is False. |
|
Server Setting |
False |
|
|
|
||
|
Org Setting |
False |
EM will not automatically apply mitigations to any Exchange server when the Org setting is False. |
|
Server Setting |
True or False |
|
If you disable mitigations on a server or for the entire organization, you can still use the EOMT to manually apply any available mitigations for new security vulnerabilities to your server until the applicable SU is installed on the server.
Blocking Mitigations
An admin can block mitigations from being reapplied using the MitigationsBlocked parameter. For example, to block a single mitigation named M1, you can use:
Set-ExchangeServer -Identity <ServerName> -MitigationsBlocked @(“M1”)
To block multiple mitigations named M1 and M2, you can use:
Set-ExchangeServer -Identity <ServerName> -MitigationsBlocked @(“M1”, “M2”)
To remove the M2 mitigation from a block list containing M1 and M2, you can use:
Set-ExchangeServer -Identity <ServerName> -MitigationsBlocked @(“M1”)
To remove all mitigations from the block list, you can use:
Set-ExchangeServer -Identity <ServerName> -MitigationsBlocked @()
After a mitigation is removed from the block list, it will be reapplied by the EM service. You can manually reapply a mitigation without waiting for the EM service’s next run by restarting the EM service. After the EM service restarts, the first run to check for and apply mitigations occurs after 10 minutes.
Mitigation Removal
When a mitigation has been blocked by an admin, or when it is no longer required (as in the case of a CU or SU update), the admin must manually remove applied mitigation actions to reverse their effects.
For example, if mitigation M1 is no longer relevant after installing the latest SU, the EM service will stop applying M1 on an hourly basis and this mitigation will disappear from the list of applied mitigations, but the mitigation would still remain configured.
The steps to remove a mitigation that’s no longer needed depend on the mitigation itself. For example, if as part of a mitigation an Exchange service is stopped and set to disabled, then removing the mitigation involves starting the service and setting it to automatic startup. To remove an IIS rewrite rule mitigation, an admin can delete the rule from the appropriate web site using IIS Manager. As illustrated in the following figure, the EM service creates IIS rewrite rules with a prefix of “EEMS <Mitigation ID> <Description>” making them easy to identify.
After identifying the appropriate IIS URL rewrite rule, an admin can manually delete the rule.
Reapplying a Mitigation
If an admin removes a mitigation but does not block it, the EM service will reapply the mitigation when it performs its hourly check for new mitigations. You can manually reapply a mitigation without waiting for the EM service to check for mitigations by stopping and restarting the EM service. After restart of EM service, the first run to check and apply mitigation occurs after 10 minutes.
Viewing Applied and Blocked Mitigations
If a mitigation is successfully applied, fails to be applied, or is blocked by an admin, that will be tracked via Applied Mitigations and Blocked Mitigations. Tracking this information is part of mitigation management. For example, after a mitigation has been successfully applied, the admin still needs to install the appropriate SU for the underlying vulnerability. After the SU is installed, the mitigations are no longer needed, and the admin must remove them (and the EM service will not reapply them).
Once mitigations are applied to a server, an admin can view the applied mitigations. For example:
Get-ExchangeServer -Identity <ServerName> | fl name, MitigationsApplied
Name : Server1
MitigationsApplied : {M1, M2, M3}
You can see the list of applied and blocked mitigations across your environment. For example:
Get-ExchangeServer -Identity <ServerName> | fl name, MitigationsApplied, MitigationsBlocked
Name : Server1
MitigationsApplied : {M1, M3}
MitigationsBlocked : {M2}
Name : Server2
MitigationsApplied : {M1, M2}
MitigationsBlocked : {M3}
You can see the list of applied and blocked mitigations on a per-server basis, as well. For example:
Get-ExchangeServer -Identity <ServerName> | fl name, *Mitigations*
Name : Server1
MitigationsEnabled : True
MitigationsApplied : {M1, M3}
MitigationsBlocked : {M2}
You can export the list of applied mitigations and their descriptions to a CSV file using the Get-Mitigation.ps1 script available in the Scripts folder. For example:
.\Get-Mitigation.ps1 -Identity <Server> -ExportCSV “C:\temp\CSVReport.csv”
Auditing and Logging
As mentioned previously, mitigation activity is logged in the Windows Event Log. For example, events 1005 and 1006 with a source of MSExchange Mitigation Service will be logged for successful actions such as when a mitigation is applied, and event 1008 with the same source will be logged for any errors that are encountered, such as when the EM service cannot reach the OCS.
In addition to logging mitigation activity, the EM service also logs details about service startup, shutdown, and termination (like all services running on Windows), as well as details of its actions and any errors encountered by the EM service. You can use Search-AdminAuditLog to review actions taken by an admin, including enabling and disabling automatic mitigations and diagnostic data.
The EM service also maintains a separate log file in the in the “V15\Logging\MitigationService” folder under the Exchange Server installation directory. This log details the tasks performed by the EM service, including fetching mitigations, parsing mitigations, and applying mitigations, as well as details about the information sent to the OCS if sending diagnostic data is enabled.
Checking Connectivity with the OCS
An admin can verify that an Exchange server has connectivity to the OCS using the Test-MitigationServiceConnectivity.ps1 script (available in the Scripts folder) from the Exchange server. The script attempts to connect to https://officeclient.microsoft.com/getexchangemitigations.
If the connection is successful, the output is:
Result: Success.
Message: The Mitigation Service endpoint is accessible from this computer.
If the connection is unsuccessful, the output is:
Result: Failed.
Message: Unable to connect to the Mitigation Service endpoint from this computer.
To learn about connectivity requirements, see https://aka.ms/HelpConnectivityEEMS
Summary
Maintaining the security of on-premises workloads is a team effort. As the customer, you are responsible for keeping your systems up-to-date and secure. As your software vendor, we are here to provide an array of tools to help you do that.
FAQ
Q: What is Exchange Server Emergency Mitigation?
A: Exchange Server Emergency Mitigation is a new feature in Exchange Server introduced in the September 2021 CUs. It detects Exchange Servers that are vulnerable to one or more known threats and applies temporary mitigations until the admin can install the available SU.
Q: What does Exchange Server Emergency Mitigation do?
A: This feature runs as a Windows service, and it checks the Office Config Service for available mitigations hourly. If a mitigation is available, the EM service downloads it and automatically applies it to the server.
Q: What is a Mitigation?
A: A mitigation is an action or set of actions used to secure an Exchange server from a known threat. If a security threat becomes known to Microsoft and we create a mitigation for the issue, that mitigation can downloaded to the Exchange server, which can automatically implement the pre-configured settings. Mitigations are sent in a signed XML file that contains configuration settings for mitigating a security threat.
Q: If I disable sending optional data to Microsoft, will the EM service still automatically apply mitigations?
A: Yes. Automatic mitigation by the EM service does not require the sending of data to Microsoft. If sending data to Microsoft is not enabled, the EM service will function normally.
Q: Will Microsoft release mitigations for every vulnerability that will be eventually fixed in an SU?
A: No. Our plan is to release mitigations only for the most severe security issues, such as issues that are being actively exploited in the wild. Because applying mitigations may reduce server functionality, we plan on releasing mitigations only when the highest impact or severity issues are found.
Posted at https://sl.advdat.com/2Zvthev
