Centralized Mail Transport (CMT) has been in Exchange Online for a while, and we often see customers using it without even realizing why. Because CMT introduces message routing complexity, increases the needed bandwidth because of additional message hops, and creates a dependency on the on-premises Exchange deployment, we thought it would be good to share a deep dive on CMT to help you figure out if you really need it.
Consider a scenario where your email domain’s MX record is pointed to Exchange Online, which is our recommended configuration, and you have an on-premises solution for DLP, encryption, journaling, etc., inbound email to Exchange Online must be routed through on-premises before being delivered to an Exchange Online hosted mailbox. For similar reasons, you may want to route all outbound messages sent to the Internet from Exchange Online via Exchange on-premises. CMT allows you to achieve both scenarios, routing all messages from Exchange Online mailboxes through Exchange on-premises before they're delivered to the Internet, and routing incoming Internet messages through Exchange on-premises before being delivered to an Exchange Online recipient. Although using CMT in this example is valid, we still encourage you to use Exchange Online which has DLP, encryption, and journaling features out of the box that allow you to avoid the complex mail routing needed with third-party systems and CMT. On the other hand, there are scenarios where CMT might be the only option for specific transport needs that Exchange Online does not provide, such as a custom transport agent or the address rewriting performed by the Edge Transport role.
While the concept of CMT is simple, there are few nuances covered below. One important aspect of CMT is that it will not send the message back to Exchange on-premises if the message already came from Exchange on-premises. In other words, if the message is “Originating” from on-premises (when the message is attributed to your tenant through an Inbound Connector of OnPremises type; see Office 365 Message Attribution for more details) CMT will not come into play because that would cause a mail loop between Exchange on-premises and Exchange Online.
Scenarios not covered by CMT:
- Messages originating from Exchange on-premises
- Messages sent between two Exchange Online mailboxes (more later in the article)
- Messages forwarded by Exchange Online (more on this later, too)
- Messages that trigger a transport rule redirecting it via Criteria Based Routing (CBR) through a specific Outbound Connector; CBR has preference over CMT
Centralized Mail Trasport vs Criteria Based Routing
CBR, also known as Conditional Mail Routing, is a mechanism designed to route mail matching certain criteria through a specific outbound connector. Although it can be used to perform the same job as CMT, CBR will not prevent a mail loop like CMT does out of the box. Specifically, if you create an Exchange Transport Rule (ETR) in Exchange Online to perform CBR, the messages sent to the on-premises system will come back to Exchange Online, which could again trigger the ETR, creating a loop until Exchange shuts it down due to an exceeded hop count. So, you might wonder why someone would choose CBR instead of CMT. There are two valid reasons:
- To force messages from one Exchange Online mailbox to another Exchange Online mailbox in the same tenant through on-premises; and
- To mix different routes using ETR conditions to force messages through a specific Outbound Connector.
Before starting the ETR creation, we need to create an Outbound Connector. To force messages from an Exchange Online mailbox to another Exchange Online mailbox through on-premises, the following connector properties are necessary:
- ConnectorType: OnPremises
- SmartHosts: <On-premises Public IP>
- IsTransportRuleScoped: True
- CloudServicesViaOnPremises: True
Once the Outbound connector is created, the following ETR can be created:
Apply this rule if:
- The sender is located: Inside the organization
AND
- The recipient is located: Inside the organization
Do the following:
- Use the following connector: <Connector Name>
Except if:
- A message header matches X-OriginatorOrg values: <All your accepted domains>
The ETR exception is key to prevent mail looping. Basically, if the ETR finds the X-OriginatorOrg header with any accepted domain, it will not be applied because it means that the message already came from on-premises.
NOTE: Do not use this rule if you have only cloud mailboxes, and be aware that any issues related to cross-premises header promotion or tenant attribution could cause unexpected behaviors.
We can leverage any ETR predicate to route messages through a specific Outbound Connector but remember that although we are explaining how CBR can address some CMT limitations, we do not encourage this approach. As previously mentioned, CMT was designed to avoid mail loops. If you use CBR, it is your responsibility to create a proper ETR to avoid mail looping or other mail flow issues.
Centralized Mail Trasport vs Partner Outbound Connector
A Partner type of Outbound Connector is intended to scope outbound messages to external domains through a specific smarthost, which usually may be a partner organization where you want to enforce TLS or an antispam service. Is not unusual to see customers claiming to have CMT enabled when actually a Partner Outbound Connector type is configured with the RecipientDomains wildcard “*” value. First things first, CMT cannot be enabled on Partner Outbound Connector type. The Partner Outbound Connector with the RecipientDomains “*” value will only route messages with external recipients through the Partner Outbound Connector.
It’s also important to note that if CMT is enabled, no Partner Outbound Connector type will be evaluated, even if the connector’s RecipientDomains has a better match. The only exception is Originating messages that as previously mentioned are ignored by CMT; in that case Partner Outbound Connectors can be evaluated. The following flow chart describes the order that connectors pick up the message to send:
(You can download the flow charts from this blog post in the ZIP file attached to this post)
Although not recommended, outbound messages to external recipients from Exchange Online to the Internet can be routed through Exchange on-premises using a Partner Outbound Connector with the RecipientDomains of “*”. As previously mentioned, CMT has a logic to prevent mail loops with on-premises, but this Partner Outbound Connector doesn’t. Additionally, you still must keep an Outbound Connector type of OnPremises created by Hybrid Configuration Wizard for the cross-premises mail flow. We don’t support hybrid mail flow between Exchange Online and on-premises mailboxes through a Partner Outbound Connector due to the CloudServicesMailEnabled parameter that cannot be enabled on Partner Outbound Connector. That would result in cross-premises headers not being promoted and might cause several unexpected behaviors. Thus, in that scenario, be sure that Exchange Online and on-premises mail flow goes through the OnPremises Outbound Connector, and outgoing message to external recipients goes through the Partner Outbound Connector, as in the following example:
NOTE: If you are using Partner Outbound Connector with the RecipientDomains of “*” pay close attention to the “Commonly misconfigured scenarios” section below which covers an issue related to this configuration.
Forwarding and CMT
As mentioned in the “Scenarios not covered by Centralized Mail Transport” section, when an email message is sent through the on-premises system via an OnPremises Inbound Connector to a mailbox hosted in Exchange Online (for example, bill@contoso.com) and the Exchange Online recipient has a forwarding SMTP address that's set to an external recipient (bill@tailspintoys.com), the CMT will not route the forwarded message back through on-premises.
The forwarded message is created and sent to the external recipient as an exact copy of the original message. Mail routing logic sees that this new message originated in the on-premises environment and therefore to avoid mail loops it doesn't send the message back to the on-premises environment. Instead, it's routed by default directly to the external recipient domain from EOP via an MX record lookup. This behavior is by design and documented.
The workaround to have Exchange Online route the mail back through on-premises is to create a Partner Outbound Connector with the RecipientDomains of “*” and smarthosts set to Exchange on-premises servers. This will result in normal outbound messages from Exchange Online being routed to on-premises through CMT, but forwarded messages will be routed through the Partner Outbound Connector:
Journaling and CMT
A common requirement of many customers is to leverage CMT but use a different route for journaled messages. You may think that a scenario like this could be easily addressed by CBR, but that’s not true. CBR relies on ETR mechanisms to force a specific route, but journaled messages (as with other kinds of system messages) cannot be controlled by transport rules. In other words, system messages bypass the ETR in the transport pipeline.
The trick in this case may be simpler than you thought: you need a “double CMT.” In other words, you need to create a second OnPremises Outbound Connector with RouteAllMessagesViaOnPremises enabled to route journaled messages to your third-party journal system by scoping by the RecipientDomains parameter to match the unique email domain or subdomain used by the journal mailbox:
In the above example journaling.fabrikam.com is the email domain that corresponds to the external mailbox defined to receive journal reports:
Centralized Mail Flow Agent
The Edge Transport role has a particular Transport Agent called “Centralized Mail Flow Agent” (CMFA). Before digging into how the CMFA works, let’s have a look at a scenario where CMFA comes into play.
Consider the following situation: an Edge Subscription is properly configured and CMT or CBR is enabled. Once an Exchange Online originating message with external recipient is received by the Edge Transport server, the message is routed to the appropriate Mailbox server. If Edge’s Send Connectors have been created through the Edge Subscription process, the Send Connector “EdgeSync - Inbound to Default-First-Site-Name” will contain the AddressSpaces value like this: “--” (two dashes); this value represents all Accepted Domains. Edge Transport will send only messages where the recipient domain is listed in Accepted Domains in the Exchange organization. But what about CMT being enabled and Edge Transport receiving originating messages where the recipient is external? Well, that’s exactly the scenario where the CMFA comes to the rescue.
Basically, the CMFA will perform two actions:
- Stamp the X-MS-Exchange-Organization-CentralizedMailFlowOverrideSet header with null value; and
- Set the RoutingOverride property to the default Accepted Domain.
A header will be stamped as the first action for future detection, meaning that if the header is present, no more routing override will be needed. In other words, this header will prevent mail loops if the email comes back to the Edge Transport server while going out.
The second step is leveraging the RoutingOverride class to force all those originating messages through the “EdgeSync - Inbound to Default-First-Site-Name” Send Connector. RoutingOverride is raised on the OnResolvedMessage event of the categorizer, and it overrides the default routing behavior by using the SetRoutingOverride method to set the RoutingDomain property per-recipient (in the case of the CMFA, the value will be default Accepted Domain). This will force the message to go through a specific Send Connector, which in this case will be the “EdgeSync - Inbound to Default-First-Site-Name” that contains the “--” AddressSpace value.
The CMFA cannot be disabled or managed, so be sure that the Edge Transport server is properly subscribed. If the Edge Transport server is not subscribed to the Exchange organization in a hybrid scenario, you will face mail looping and other unexpected issues.
Commonly misconfigured scenarios
Partner Outbound Connector used with “*” RecipientDomains causing mail loop
The following example describes the mail loop between the contoso.com tenant and on-premises when the recipient domain tailspintoys.com is a hosted mailbox in the same EOP forest:
To avoid this mail loop, ensure that Exchange on-premises is presenting a unique TLS certificate to send messages to the Internet. The TLS certificate name cannot match the TlsSenderCertificateName parameter of your OnPremises Inbound Connector. If the TlsSenderCertificateName value is the same and the recipient’s tenant is in the same EOP forest, the message will be attributed to your tenant instead of being attributed to the recipient’s tenant as described in the FAQ 6-b in this earlier post. If your OnPremises Inbound Connector is using SenderIPAddresses rather than TlsSenderCertificateName, things get more complicated. Unlike the TLS certificate, there is no way to scope a unique IP to a specific Send Connector. In that case, it’s highly recommended to use TlsSenderCertificateName rather than SenderIPAddresses with a unique TLS certificate.
If you are not using Exchange on-premises and the mail loop is caused by a third-party MTA attributing the message to the same recipient's tenant, you can convert the Exchange Online Inbound Connector from OnPremises to Partner, which would be enough to avoid wrong attribution as Partner Inbound Connectors are not evaluated in the tenant attribution processes.
EOP not honoring the previous verdict when CMT is enabled
Imagine the following scenario: inbound messages are entering EOP and are marked as spam, spoofing, or phishing. If CMT is enabled and none of these categories have the quarantine action, these messages will go through on-premises and come back to EOP because the user’s mailbox is hosted on Exchange Online. But once the round-trip is made, the message is delivered to the user’s inbox, even if the previous result was spam, spoofing, or phishing.
If cross-premises header promotion is working properly, you should expect the message being marked as CAT:SPOOF in the first EOP entry:
X-Forefront-Antispam-Report-Untrusted: SCL:5;IPV:NLI;SFV:SPM;CAT:SPOOF;SFTY:9.25;DIR:INB
On the round-trip back into Exchange Online, you should see some different header values:
X-Forefront-Antispam-Report: SCL:5;IPV:NLI;SFV:SKS;CAT:SPM;DIR:INB
Note a few differences between the first anti-spam entry and the round-trip anti-spam headers. The first is that we have “-Untrusted” at the end of the header, which means that the existing X-Forefront-Antispam-Report header from the first entry was renamed to X-Forefront-Antispam-Report-Untrusted.
The two other important differences are the SFV:SKS and the CAT:SPM values. SKS means that a previous SCL exists, so we do not rescan the message again and the first SCL will be honored. Regarding the change from CAT:SPOOF to CAT:SPM, this is an expected behavior, so in this case the SPAM action will be taken instead the SPOOF action.
If the previous anti-spam scanning results are not honored, it might be due to cross-premises header promotion or the tenant attribution process. Specifically, the header promotion from X-MS-Exchange-Organization-* headers to X-MS-Exchange-CrossPremises-* and vice versa, or the message attribution to an OnPremises Inbound Connector. This article provides a deep dive on how cross-premises header promotion works.
The reason why header promotion issues can affect the round-trip is related to the header firewall. If we do not receive the originating message with the org headers properly promoted, the SCL will be striped, and subsequently the first SCL verdict will not be honored, thus antispam filters may work unexpectedly.
CMT enabled during cross-tenant mailbox migration (preview)
Although this is not exactly a misconfiguration as CMT could also be properly configured, having CMT enabled on the target tenant during the cross-tenant migration might cause a mail loop that will end up in 5.4.6 or 5.4.14 Routing loop detected error.
The “misconfiguration” may happen because the mailbox move performed by the Mailbox Replication service will not update the target on-premises Active Directory object (specifically the targetAddress attribute aka ExternalEmailAddress), which keeps pointing to the source tenant.
That said, disabling CMT on the target tenant is probably the best option to avoid any data loss and mail looping. Once the migration is completed and you have properly updated the targetAddress attribute of all migrated mailboxes, you can safely re-enable CMT.
Centralized Mail Transport FAQ:
Does Microsoft support CMT routing to a third-party mail system?
We support this only for third-party MTAs which don’t send the messages back to EOP, such as journaling systems. The reason we don’t support this is related to header promotion; it means that only an on-premises Exchange Mailbox role or an Edge Transport role has all necessary capabilities to promote the cross-premises headers (such as “AuthAs:Internal”) and keep them across the transport pipeline to send the messages back to Exchange Online honoring the authentication decision.
Can I use Centralized Mail Transport with Microsoft 365 Groups on-premises?
Our official statement is that if CMT is enabled, then external email to Microsoft 365 Groups will be rejected. The technical reason behind this is not directly related to CMT but rather Azure AD Connect. In short, Azure AD Connect doesn’t support the write-back of the msExchRequireAuthToSendTo attribute, so this attribute is updated to TRUE on each sync cycle. This means that external or even internal but non-authenticated messages (from printers or applications) will be rejected by Exchange on-premises with the following NDR: '550 5.7.133 RESOLVER.RST.SenderNotAuthenticatedForGroup; authentication required; Delivery restriction check failed because the sender was not authenticated when sending to this group'.
Can I disable or enable CMT manually instead of using the Hybrid Configuration Wizard?
HCW is preferred, but you can also use Exchange Online PowerShell.
To enable using Exchange Online PowerShell:
Set-OutboundConnector -Identity “<Hybrid Outbound Connector name>” -RecipientDomains * -RouteAllMessagesViaOnPremises:$True
To disable using Exchange Online PowerShell:
Set-OutboundConnector -Identity “<Hybrid Outbound Connector name>” -RecipientDomains “<All your accepted domains separated by comma>” -RouteAllMessagesViaOnPremises:$False
I want to disable CMT, are there any caveats that I should be aware of?
By disabling CMT, CBR, or Partner Outbound Connector with the RecipientDomains value of “*” you are removing complexity from your mail flow environment as outbound messages will route to the Internet from EOP via MX record lookups. Just be sure to check the following things:
- Verify that the SPF record has “include:spf.protection.outlook.com”.
- Check if any on-premises ETRs should be replicated to Exchange Online.
- Although rarely, on-premises Send Connectors could leverage basic authentication to send to a third-party MTA through AuthenticationCredential parameter. Consider that EOP always tries Opportunistic-TLS, and you are able to use TlsAuthLevel as DomainValidation but be aware that there is no option to leverage basic authentication on an Exchange Online Outbound Connector.
- No custom Transport Agent can be installed in Exchange Online. If you rely on a third-party service, verify with your provider that the service can meet the following requirements: Integrate Microsoft 365 or Office 365 with an email add-on service.
- Exchange Online doesn’t provide address rewriting. If you have the Edge Transport role performing address rewriting of RCPT TO, To or Cc you can simply point your MX record to on-premises and disable CMT. If you are rewriting other email fields, you still need CMT for the outbound messages.
- On-premises tracking logs can be retained for an unlimited time, but Exchange Online trace logs are available for a limited period. If you have compliance requirements related to log retention, be sure to collect the trace log through the available API and retain for the period that meets your requirements.
Summary
We hope that this article provides you with enough information about CMT, CBR and Outbound Connectors. Please use the comment section to ask questions or provide suggestions!
Special thanks to Arindam Thokder and Dan Sheehan for the technical contribution.
Posted at https://sl.advdat.com/3q8p5g2
