Recognizing a disaster

Failures that require disaster recovery

The disaster recovery procedure is an extreme measure that should be used only if the failure disrupts all communication from one MetroCluster site to the other for a prolonged period of time.

The following are examples of disasters that could cause such a failure:

Fire
Earthquake
Prolonged power outages at a site
Prolonged loss of connectivity from clients to the filers at a site

Ways to determine whether a disaster occurred

It is critical that you follow some predefined methodical procedure to confirm that a disaster really occurred. The procedure should include determining the status of the supposed disaster site by

Using external interfaces to the filer, such as the following:
- Ping
- Remote shell
- FilerView
Using network management tools to verify connectivity to the disaster site
Physically inspecting the disaster site, if possible

You should declare a disaster only after verifying that a disaster occurred and that service cannot be restored.

Failures that do not require disaster recovery

If the failure is one in which the cluster relationship is reestablished after fixing the problem, you do not want to perform the disaster recovery procedure.

Do not perform the disaster recovery procedure for the following failures:

A failure of the cluster interconnect between the two sites. This can be caused by the following failures:
- Failure of the interconnect cable
- Failure of one of the VI cluster adapters
- If using switches, a failure of the SFP connecting a cluster node to the switch
With this type of failure, both nodes of the cluster remain running. Automatic takeover is disabled because Data ONTAP cannot synchronize the nonvolatile RAM (NVRAM) logs. After you fix the problem and reestablish the connection, the cluster nodes resynchronize their NVRAM logs and the cluster returns to normal operation.
The storage from one site (site A) is not accessible to the cluster node at the other site (site B). This can be caused by the following failures:
- Failure of any of the cables connecting the storage at one site to the cluster node at the other site or switch
- If using switches, failure of any of the SFPs connecting the storage to the switch or the cluster node to the switch
- Failure of the Fibre Channel adapter on the cluster node
- Failure of a storage disk shelf (loop resiliency circuit, power, access to disk shelves, and so on)
With this type of failure, you see a "mailbox disk invalid" message on the console of the filer that cannot see the storage. After you fix the problem and reestablish the connection, the cluster returns to normal operation.
If you are using switches, the Inter-Switch Link (ISL) between the two switches fails.
With this type of failure, both nodes of the cluster remain running. You see a "mailbox disk invalid" message because a filer at one site cannot see the storage at the other site. You also see a message because the two filers cannot communicate with each other. After you fix the problem and reestablish the connection, the cluster nodes resynchronize their NVRAM logs and the cluster returns to normal operation.