Since the introduction of DAG into Exchange Server architecture, the DR situations became more easy and comfortable to deal with. On this article I have tried to pen down a simple and straight forward process on how to deal with DR scenario.
My LAB Servers:
Exchange Servers (4 nos): EX16-01, EX16-02, EX16-03, EX16-Dr1
AD Servers: 2010adc1 & 2010adc2 (both work as Witness/Alternate Witness servers respectively)
DAG Name: DAG02 (Please note I have configured Exchange Server 2016 DAG without cluster object)
Witness Server Information:
WitnessServer : 2010adc1.lab.ed.com
WitnessDirectory : c:\DAG02
Alt Witness Server : 2010adc2.lab.ed.com
Alt Witness Dir : c:\DAG02
Scenario: 2 Servers in Primary Datacenter & 2 Servers in Secondary datacenter. I plan to bring down 2 servers EX16-02 & EX16-03 and restore the DAG with EX16-01 & EX16-dr1 server.
Step1: Bring down Servers as planned (Or assume both the server are crashed due to some unforeseen issues at the Primary Datacenter, and we expect these servers to come up after restoring the services at Primary data center).
Check the cluster node status,
Step2: Terminate the partially or failed Servers (in our case the server which are OFF)
[PS]>Stop-DatabaseAvailabilityGroup -Identity DAG02 -MailboxServer EX16-02 –ConfigurationOnly
[PS]>Stop-DatabaseAvailabilityGroup -Identity DAG02 -MailboxServer EX16-03 –ConfigurationOnly
You may verify the DAG statistics by executing the following command.
[PS]>Get-DatabaseAvailabilityGroup dag02 | fl Name,st*,prima*,oper*,*wit*
Step3: Stop the cluster service on all the nodes which are currently running and will be part of the cluster during the DR operations. It is important to terminate/switch off all failed partially failed servers in the DAG before we proceed to next step.
Note: Run the above command on all the running members, in our case EX16-01 & EX16-dr1.
Step4: Restore the DAG using Restore-DatabaseAvailabilityGroup command
[PS]>Restore-DatabaseAvailabilityGroup -Identity DAG02
Note: if you have not configured the DAG with alternate witness server prior to this operations, you may provide the alternate witness server values along with the Restore DAG command.
The restore DAG operation may take extended period of time depends on the server resources and network latencies. Please do patient until it returns the results.
Step5: Remove the restriction on DB activation
With the above command, we have finished activating the DR mailbox servers. Now we can remove the activation restriction on the databases, so that the database will start mounting automatically.
[PS]>Get-MailboxDatabaseCopyStatus | Resume-MailboxDatabaseCopy
Validate the status, you should find mailbox databases gets mounted automatically in a while.
You may also verify the cluster node status, by now only the nodes which are active will be listed.
Let’s assume both the failed servers are now back online. Please verify that the cluster services are in disabled & stopped status on both these servers. BTW, restoring services is more easy and quick than failover/switchover procedure.
Step1: Put the Mailbox servers in the restored primary datacenter into a started state.
[PS]>Start-DatabaseAvailabilitygroup -Identity DAG02 -mailboxServer EX16-02
[PS]>Start-DatabaseAvailabilitygroup -Identity DAG02 -mailboxServer EX16-03
The warnings can be ignored safely.
Step2: Run Set-DatabaseAvailabilityGroup to commit the members back to cluster.
[PS] >Set-DatabaseAvailabilityGroup -Identity DAG02
During the course of this operation, the default settings of clusters will be brought back such as witness server, directory and other site settings if any. After the Mailbox servers in the primary datacenter have been incorporated into the DAG, it will take some time to synchronize the database copies.
With that we finished fail back procedure, as the database will be mounted automatically to the primary datacenters as we have not changed its activation preferences.
You may also force to mount database by activating it through EMC or EAC.
Please share you experience in the form of comments, if any.