Monthly Archives: September 2012

Sep 19 2012

Procedure for Exchange 2010 DAG – Datacenter Failover

Brief

In Our Scenarios we had 10 Mailbox Server, 5 in each site & 2 Public Folder Server one in each site. We had two witnesses Server, the primary was set in DC1 and the alternate was set in DC2, so our primary Site was DC1.

This article is based on When we have Datacenter Activation Coordination Protocol is Enabled (DACP).

Step1. Ensuring that Primary DC Outage has been Declared

Ping Mailbox Server

Ping HUB Server

Ping CAS Server

Step2. Stopping the DAG from the Secondary DC for Primary DC Mailbox Server

Note:

  • Ensure that you have Full Permission on the Exchange ORG, there should not be any restriction using RBAC.
  • Before running the below command open the Exchange Management Shell “Run as administrator” otherwise the cmdlets won’t load to the PowerShell you may get an error message.
  • This has to be ran from any of the Exchange Server From the Secondary DC

Stop-DatabaseAvailAbilityGroup –identity DAG1 -Mailboxserver EXM01.contoso.com -configurationonly

Stop-DatabaseAvailAbilityGroup –identity DAG1 -Mailboxserver EXM02.contoso.com -configurationonly

Stop-DatabaseAvailAbilityGroup –identity DAG1 -Mailboxserver EXM03.contoso.com -configurationonly

Stop-DatabaseAvailAbilityGroup –identity DAG1 -Mailboxserver EXM04.contoso.com -configurationonly

Stop-DatabaseAvailAbilityGroup –identity DAG1 -Mailboxserver EXM05.contoso.com –configurationonly

Step3. Stopping the Cluster Service on every DAG-Member of the secondary data-center

Note:

You might see sometime it will result that Cluster Service is already stopped, but no, check using the Services Console, Ensure Cluster Service is in stopped state.

Net stop clussvc

Step4. Reactivate the mailbox-servers in the standby data

Restore-databaseavailabilitygroup –identity DAG1

Note:

Most of the time you will see database will start mounting after executing this command successfully, and also this will start the cluster service on all the DAG members in the secondary DC. If this happens so then skip the step 5, if cluster service has not started perform the step.

Step5. Starting the Cluster Service on every DAG-Member of the secondary data-center

Net start clussvc

or

Open the services console and choose the Cluster Service and start it.

Step6. All the DB should be mounted to the respective Secondary DAG members

Note:

All the DB should be switched to the respective DAG member successfully. If mailbox-databases don’t switch automatically run the script to move databases to passive

Resume-MailboxDatabaseCopy ‘DATABASENAME\SERVERNAME’

or

Mount-Database -Identity ExchangeServerFQDN\DatabaseName

Step7. Check the Status of the DB

Check the DB Copy Status and Ensure they all are healthy. In My Case I had DB Created with Standard Name DB1 to DB72, So I used the below PowerShell to check the copy of all in one shot.

Get-MailboxDatabaseCopyStatus -Identity DB* | select name, status, SelectcontentIndexState | sort Status | ft –auto

Note:

Mount the databases if not done automatically.

Step8. Move OAB Generation to to Secondary Site Public Folder Server

Move-OfflineAddressBook -Identity “Offline Address Book” -Server EXP11

Step9. Do the Final check for Functionalities i.e.

Note:  

While accessing the Public Folder using Outlook you might see the outlook hanging but that’s ok, you can ignore it as once the connection get thru with new Public Folder Server, you won’t find the slowness.

  • Email Flow (Internal/External):
  • PUB Folder Access 
  • ActiveSync:
  • All DB: Mounted
  • Free/Busy:
  • OAB:
  • OWA:
  • POP3:
  • IMAP:
  • Blackberry:

Step9. Primary DC Switchback Steps

Note:

You got to know that the Network Connectivity has been resumed to the primary DC.

Also ensure that you got the final instruction that to proceed with Primary Datacenter Restore for exchange.

Ping Primary Mailbox Server

Ping Primary HUB Server

Ping Primary CAS Server

Step10. Ensure Cluster service has been started on Primary Datacenter Server, if not start it

Note:

You will see that Cluster Service is in Disabled state on all the Primary Datacenter in most cases the cluster service won’t start 

First enable the service by going to services.msc console and then start it

Or

Net start clussvc

If you are getting the error while starting the Cluster Service, then we need to perform cleanup by running the below command.

 

 

Cluster Node EXM01.contoso.com /forcecleanup

Cluster Node EXM02.contoso.com /forcecleanup

Cluster Node EXM03.contoso.com /forcecleanup

Cluster Node EXM04.contoso.com /forcecleanup

Cluster Node EXM05.contoso.com /forcecleanup

Step11. Start the DAG for all the Failed Server

Start-DatabaseAvailabilityGroup -MailboxServer EXM01.contoso.com

Start-DatabaseAvailabilityGroup -MailboxServer EXM02.contoso.com

Start-DatabaseAvailabilityGroup -MailboxServer EXM03.contoso.com

Start-DatabaseAvailabilityGroup -MailboxServer EXM04.contoso.com

Start-DatabaseAvailabilityGroup -MailboxServer EXM05.contoso.com

Step12. Then run the below command from any of the Exchange server

Set-DatabaseAvailabilityGroup –Identity DAG1

Step13. Move the OAB Generation server back to the EXP01 as it was earlier

Move-OfflineAddressBook -Identity “Offline Address Book” -Server EXP01

Step14. Redistribute the Mailbox Database copy across the Primary and Secondary DC

Note:

Ensure all the DBs are mounted and second copies are in health state.

Ensure all the nodes are up and showing good in the Failover Cluster Manager

In My Case I used mine one. You Can use your Own Script to Distribute The DB

Step15. Check all the functionalities

  • Email Flow (Internal/External)
  • PUB Folder Access:
  • ActiveSync:
  • All DB: Mounted
  • Free/Busy:
  • OAB:
  • OWA:
  • POP3:
  • IMAP:
  • Blackberry:

 Cheers,