You can use cluster failover, a high-availability data clustering solution, to further increase the uptime of storage systems. It protects against appliance controller failure by transferring the data service from the failed storage system to its partner storage system. In addition to appliance head failure, cluster failover can also protect against other hardware failures, such as network interface cards, Fibre Channel-Arbitrated Loops (FC-AL loops), and LRC, ESH, ESH2, or AT-FCX modules. Cluster failover is also an effective tool for reducing planned downtime of one of the nodes.
SPOF represents the failure of a single hardware component that can lead to loss of data access or potential loss of data. SPOF does not include multiple/rolling hardware errors, such as double disk failure, dual LRC, ESH, ESH2, or AT-FCX failure, etc.
All hardware components have demonstrated very good reliability with low failure rates. If a hardware component fails, such as an appliance controller or adapter, you can use cluster failover to provide continuous data availability and preserve data integrity for client applications and users.
The following table shows which hardware components are SPOFs, and how cluster failover can eliminate these SPOFs to improve data availability.
Hardware components
|
SPOF
|
How cluster failover eliminates SPOF
|
|---|
Single storage system
|
Clustered storage system
|
|---|
|
Storage system controller
|
Yes
|
No
|
If a storage system controller fails, the storage system automatically fails over to its partner node. The partner (takeover) node serves data for both of the nodes.
|
|
NVRAM
|
Yes
|
No
|
If an NVRAM adapter fails, the storage system automatically fails over to its partner node. The partner (takeover) node serves data for both of the nodes.
|
|
CPU fan
|
Yes
|
No
|
If the CPU fan fails, the storage system gracefully shuts down and automatically fails over to its partner node. The partner (takeover) node serves data for both of the nodes.
|
|
Multiple NICs with VIFS (virtual interfaces)
|
No
|
No
|
If one of the networking links within a VIF fails, the networking traffic is automatically sent over the remaining networking links on the storage system. No failover is needed in this situation.
If all the NICs in a VIF fail, the node automatically fails over to its partner node, if failover is enabled for the VIF.
|
|
Single NIC
|
Yes
|
No
|
If a NIC fails, the node automatically fails over to its partner node, if failover is enabled for the NIC.
|
|
FC-AL adapter
|
Yes
|
No
|
If an FC-AL adapter for the primary loop fails, the partner node attempts a failover at the time of failure.
If the FC-AL adapter for the secondary loop fails, the failover capability is disabled, but both nodes continue to serve data to their respective applications and users, with no impact or delay.
|
|
FC-AL cable (appliance-to-shelf, shelf-to-shelf)
|
Yes
|
No
|
If an FC-AL loop breaks, it could lead to a failover, depending on the shelf type. The clustered storage systems invoke the negotiated failover feature to determine which node is best for serving data, based on the disk shelf count.
|
|
LRC, ESH, ESH2, AT-FCX
|
Yes
|
No
|
If one of these components fails, it could lead to a failover. The clustered storage systems invoke the negotiated failover feature to determine which node is best for serving data, based on the disk shelf count.
|
|
Disk drive
|
No
|
No
|
If a disk fails, the storage system can reconstruct data from the RAID4 parity disk. No failover is needed in this situation.
|
|
Disk shelf (including backplane)
|
No
|
No
|
A disk shelf is a passive backplane with dual power supplies, dual fans, dual LRCs, and dual FC-AL loops. It is the single most reliable component in a storage system.
|
|
Power supply
|
No
|
No
|
Both the storage system head and disk shelf have dual power supplies. If one power supply fails, the second power supply automatically kicks in. No failover is needed in this situation. If both power supplies fail, the node automatically fails over to its partner node, which serves data for both nodes.
|
|
Fan (storage system or disk shelf)
|
No
|
No
|
Both the storage system head and disk shelf have multiple fans. If one fan fails, the second fan automatically provides cooling. No failover is needed in this situation. If both fans fail, the node automatically fails over to its partner node, which serves data for both nodes.
|
|
Cluster adapter
|
N/A
|
No
|
If a cluster adapter fails, the failover capability is disabled but both nodes continue to serve data to their respective applications and users.
|
|
Cluster interconnect cable
|
N/A
|
No
|
The cluster adapter supports dual cluster interconnect cables. If one cable fails, the cluster traffic (heartbeat and NVRAM data) are automatically sent over the second cable with no delay or interruption.
If both cables fail, the failover capability is disabled but both nodes continue to serve data to their respective applications and users.
|
Use this table to understand what happens when a failover event occurs, and how the various clustering configurations handle these events.
Event
|
Does the event trigger a failover?
|
Does the event prevent a future failover from occurring, or a failover from occurring successfully?
|
Is data still available on affected volume after event?
|
|---|
Cluster
|
Metro-Cluster
|
Cluster
|
Metro-Cluster
|
Single storage system
|
Cluster
|
Metro-Cluster
|
|---|
|
Single disk failure
|
No
|
No
|
No
|
No
|
Yes
|
Yes
|
Yes
|
|
Double disk failure (2 disks fail in same RAID group)
|
Yes, unless you are using Sync-Mirror or RAID-DPTM, then no.
|
No
|
Maybe. If root volume has double disk failure or if the mailbox disks are affected, no failover is possible.
|
No
|
No, unless you are using RAID-DP protection or Sync-Mirror, then yes.
|
No, unless you are using RAID-DP or Sync-Mirror, then yes.
|
Yes, with no failover necessary.
|
|
Triple disk failure (3 disks fail in same RAID group)
|
Maybe. If Sync-Mirror is being used, there won't be a takeover; otherwise, yes.
|
No
|
Maybe. If root volume has triple disk failure, no failover is possible.
|
No
|
No
|
No
|
Yes, with no failover necessary.
|
|
Single HBA (initiator) failure, Loop A
|
Maybe. If Sync-Mirror is being used, then no; otherwise, yes.
|
No
|
Maybe. If root volume has double disk failure, no failover is possible.
|
No
|
Yes, if shelf is dual-pathed, or if Sync-Mirror is being used.
|
Yes, if Sync-Mirror is being used, or if failover succeeds.
|
Yes, with no failover necessary.
|
|
Single HBA (initiator) failure, Loop B
|
No
|
No
|
Yes, unless you are using SyncMirror and the mailbox disks aren't affected, then no.
|
No
|
Yes, if shelf is dual-pathed or mirrored.
|
Yes, with no failover necessary if Sync-Mirror is being used.
|
Yes, with no failover necessary.
|
|
Single HBA initiator failure, (both loops at same time)
|
Yes, unless the data is mirrored on a different (up) loop, then no takeover needed.
|
No
|
Maybe. If the data is mirrored and the mailbox disks aren't affected, then no; otherwise, yes.
|
No
|
No, unless the data was mirrored, then yes.
|
No failover needed if data was mirrored.
|
Yes, with no failover necessary.
|
|
LRC failure (Loop A)
|
Only if multidisk volume failure or open loop condition occurs.
|
No
|
Maybe. If root volume has double disk failure, no failover is possible because this impacts the mailbox disks.
|
No
|
Yes, if shelf is dual-pathed or mirrored.
|
Yes, if failover succeeds or if data is mirrored.
|
Yes, with no failover necessary.
|
|
LRC failure (Loop B)
|
No
|
No
|
Maybe. If the data is mirrored and the mailbox disks aren't affected, automatic failover happens.
|
No
|
Yes, if shelf is dual-pathed or the data is mirrored.
|
Yes
|
Yes
|
|
ESH or AT-FCX failure (Loop A)
|
Only if multidisk volume failure or open loop condition occurs, and data isn't mirrored.
|
No
|
Maybe. If root volume has double disk failure, no failover is possible.
|
No
|
No
|
Yes, if failover succeeds.
|
Yes, with no failover necessary.
|
|
ESH or AT-FCX failure (Loop B)
|
No
|
No
|
Maybe. If SyncMirror is used, then no; otherwise, yes.
|
No
|
Yes, if shelf is dual-pathed or data is mirrored.
|
Yes
|
Yes
|
|
Shelf (backplane) failure
|
Only if multidisk volume failure or open loop condition occurs, and data is mirrored.
|
No
|
Maybe. If root volume has double disk failure or if the mailboxes are affected, no failover is possible.
|
No
|
Maybe. If data is mirrored, then yes; otherwise, no.
|
Maybe. If data is mirrored, then yes; otherwise, no.
|
Yes, with no failover necessary.
|
|
Shelf, single power failure
|
No
|
No
|
No
|
No
|
Yes
|
Yes
|
Yes
|
|
Shelf, dual power failure
|
Only if multidisk volume failure or open loop condition occurs and data isn't mirrored.
|
No
|
Maybe. If root volume has double disk failure or if the mailbox disks are affected, no failover is possible.
|
No
|
Maybe. If data is mirrored, then yes; otherwise, no.
|
Maybe. If data is mirrored, then yes; otherwise, no.
|
Yes, with no failover necessary.
|
|
Head, single power failure
|
No
|
No
|
No
|
No
|
Yes
|
Yes
|
Yes
|
|
Head, dual power failure
|
Yes
|
Yes
|
Yes, until power is restored.
|
Yes, until power is restored.
|
No
|
Yes, if failover succeeds.
|
Yes, if failover succeeds.
|
|
Cluster interconnect failure (1 port)
|
No
|
No
|
No
|
No
|
n/a
|
Yes
|
Yes
|
|
Cluster interconnect failure (both ports)
|
No
|
No
|
Yes
|
No.
Failover is possible.
|
n/a
|
Yes
|
Yes
|
|
Ethernet interface failure (primary, no VIF)
|
Yes, if set up to do so
|
Yes, if set up to do so
|
No
|
No
|
Yes
|
Yes
|
Yes
|
|
Ethernet interface failure (primary, VIF)
|
Yes, if set up to do so
|
Yes, if set up to do so
|
No
|
No
|
Yes
|
Yes
|
Yes
|
|
Ethernet interface failure (secondary, VIF)
|
Yes, if set up to do so
|
Yes, if set up to do so
|
No
|
No
|
Yes
|
Yes
|
Yes
|
|
Ethernet interface failure (VIF, all ports)
|
Yes, if set up to do so
|
Yes, if set up to do so
|
No
|
No
|
Yes
|
Yes
|
Yes
|
|
Tape interface failure
|
No
|
No
|
No
|
No
|
Yes
|
Yes
|
Yes
|
|
Heat exceeds permissible amount
|
Yes
|
Yes
|
No
|
No
|
No
|
No
|
No
|
|
Fan failures (disk shelves, storage system, or storage engine)
|
No
|
No
|
No
|
No
|
Yes
|
Yes
|
Yes
|
|
Reboot
|
No
|
No
|
No
|
No
|
Maybe. Depends on cause of reboot.
|
Maybe. Depends on cause of reboot.
|
Maybe. Depends on cause of reboot.
|
|
Panic
|
Yes
|
Yes
|
Yes
|
No
|
Maybe. Depends on cause of panic.
|
Maybe. Depends on cause of panic.
|
Maybe. Depends on cause of panic.
|
