Exchange Server 2010 Shadow Redundancy

Exchange 2007 was launched with a completely new design and product architecture and showed up with a whole new concept of bifurcated server roles which were a part of a single server installation in Exchange Server 2003. These server roles were wiz. Mailbox Server Role, Hub Transport Server Role, Client Access Server Role, Edge Transport Server Role and Unified Messaging Server role (this was a completely new feature that exchange 2007 supported. Earlier version of exchange did not have any native support to unified messaging and relied on third party products).

With these bifurcation of functionality in terms of server roles; exchange server 2007 also introduced new concepts of high availability for mailbox databases very well known as CCR, LCR, SCR. These HA features made sure that once an email is delivered to a mailbox on the database, it remains safeguarded against any data loss even in case of a complete database or server failure.

Transport Dumpster:

With highly available mailbox servers, email messages are safe once they are delivered to the database; protecting email messages in transit was another challenge. As a safety measure against any data loss for the message in transit, exchange 2007 hub transport server introduced another new feature called transport dumpster which is nothing more than a queue of messages that is maintained by the hub transport server. This queue contains the email messages which were recently delivered to the mailboxes protected by mailbox HA solution like CCR. These messages are retained in the database until they reach the age limit configured by the administrator. In case of a failover the clustered mailbox servers request all hub transport servers in the site to resubmit emails messages from transport dumpster. This mechanism prevents the loss of emails during the failover but it didn’t have the capability to handle the emails in transit between transport servers.

Shadow Redundancy uses notifications:

To make sure that least amount of data is lost in the form of in transit email messages; Exchange 2010 adds a new feature of the transport component called Shadow Redundancy. This is just like transport dumpster but except the deletion of messages from transport database is not completed until the transport server verifies that all of the database copies in DAG are successfully updated. Shadow Redundancy keeps tack of messages which have been delivered and replicated to all database copies in a DAG. This is achieved by maintaining a copy of messages being sent to database copies in a DAG and requesting a notification that it has been replicated to all database copies in DAG. The copy of messages is held in the mail.que database until the HT server receives a notification that the transaction log representing that particular message has been replicated to the primary target database and has also been successfully replicated to other copies. Once the notification is received the transaction log representing the message is truncated to make sure that the mail.que database contains only those messages which have not yet been replicated.

Shadow Redundancy Manager (SRM) and Heartbeat:

The initiating transport server issues and XQUERYDISCARD message to the target server and in response the target server returns a discard notification. This is known as the heartbeat. Once the heartbeat is received from the target; the Shadow Redundancy Manager uses this heartbeat to determine the availability of the primary target server for which the shadow messages are queued. Heartbeat time out is 300 seconds by default which is subject to 3 consecutive retries to the destination server. If the initiating server cant establish a connection to primary target within the heartbeat timeout the initiating server determines that the destination is not functional then resubmits the shadowed messages to next available server. If there is any chance of message duplication due to this approach then Exchange 2010 message de-duplication features takes the charge and removes them so that users don’t see duplicate messages in their mailboxes.

In short, Shadow redundancy is not the transport dumpster but an extension to SMTP service which works tightly with the transport dumpster. Shadow Redundancy is enabled by default and is used as far as both servers in a SMTP session support it. You can use Set-TransportConfig –ShadowRedundancyEnabled:$False. It uses discard notifications to remove only those messages from dumpster which has been delivered and successfully replicated across all database copies in a DAG.

Shadow Redundancy is the transport component and can only be managed using EMS, there is no GUI to administer it.

Microsoft Technet has a detailed article which discusses the various scenarios and the ways Shadow Redundancy works: You can read it here