SharedStorage

Understanding SharedStorage

Data ONTAP 7.1 supports SharedStorage, the ability to share a pool of disks amongst a community of NetApp storage systems, made up of two to four homogeneous NetApp FAS900 series and higher storage systems, without requiring any of the storage systems to be in a cluster. SharedStorage does not support using more than one kind of model in one community. For example, you cannot mix a FAS960 storage system with a FAS980 storage system.

You can configure SharedStorage with or without the vFiler no-copy migration functionality. If you do not want to use the vFiler no-copy migration functionality, you can create aggregates and FlexVol volumes in the community. If you want to use the vFiler no-copy migration functionality, you are restricted to creating only traditional volumes that are associated with a vFiler unit. For more information about how to use this functionality, see vFiler no-copy migration software.

The SharedStorage feature enables you to perform the following tasks:

Increase disk capacity independently of the storage systems
You can add disks (up to a maximum of 336) to any of the disk shelves and leave them unassigned. This allows you to provision spare disks amongst the community of storage systems rather than provision disks for each storage system individually.
Assign or provision individual disks across up to four storage systems to expand traditional volumes and aggregates
Assign dual paths to clustered storage systems in the community
Assign independent paths to each shelf in the community (however, you cannot daisy-chain shelves)

In addition, SharedStorage uses a standardized back-end architecture, which provides the following benefits:

Easy-to-use all-optical cabling and storage controllers
Reduced spares cost, because only one FRU is needed
Cabling flexibility, because there are multiple distance options for optical cables
Optimized bandwidth, because there are dedicated 2-Gb optical dual paths to all shelves

How SharedStorage works

SharedStorage uses external Fibre Channel hubs to connect all of the disks to all of the storage systems in the community. Each storage system can also use the hub to communicate with every other storage system. Each storage system is both an initiator and a target, so all of the storage systems can submit and receive FC requests. The storage systems can also share SES information and controls as well as state information when performing upgrades of disk firmware and other tasks.

Two hubs are connected to each storage system and each one controls an FC-AL loop, either an A loop or a B loop, to provide redundancy. Each storage system supports up to four A and four B loops. Up to six disk shelves can be directly connected to a loop switch port on each hub, so that all connected ports are logically on the same FC-AL loop.

You can set up the storage systems in the following configurations with full multiprotocol support, including NFS, CIFS, FCP, and iSCSI:

One or two clusters
One cluster with one or two single storage systems
Two to four single storage systems

The following diagram shows four storage systems, with the first two configured as a cluster. The nodes in the cluster are directly connected to each other with IB cluster adapter cables (notice that the cluster interconnect cables are not attached to the hubs).

You use software-based disk ownership to assign disks to storage systems. Each disk is dually connected, and the paths to each disk go through different disk adapters, which means that loss of a single adapter, hub, cable connection, or I/O module can be tolerated.

All of the storage systems can communicate with each other as well as all of the disk shelves and the disks in the community. Up to two storage systems can control the SES disk drives of a given disk shelf. In each shelf, at least one SES drive bay must be occupied by a disk. This allows any storage system to turn on any disk shelf's LED lights, check its environment, receive shelf status. or perform upgrades of disk firmware.

SyncMirror is supported with SharedStorage. For information about the SyncMirror rules regarding pools, see How disks are assigned to pools when SyncMirror is enabled.

How to install a SharedStorage community

Installing a SharedStorage community requires SupportEdge Premium Support service, and the Installation Service is mandatory. For information, contact your NetApp Sales representative.

The requirements for using SharedStorage include the following components:

Two to four homogenous NetApp FAS900 series storage systems with four dual-ported QLogic FC HBAs (as clustered pairs or not, in any combination)
DS14mk2 shelves, up to six shelves per storage system
ESH or ESH2 shelf modules (two per each shelf)
Emulex InSpeed 370 20-port loop switches, (two per each storage system)
Up to 336 disks (4 pairs of loops, 6 shelves, 14 disks per shelf)

For wiring information, see the Installation and Setup Instructions for NetApp SharedStorage. These instructions include the software setup procedure for booting the storage systems the first time.

After you have completed the setup procedure, verify the following:

The lights on all of used hub ports are green.
Each storage system sees all disks, which all have a primary and a secondary path (use the storage show disk -p command to display both paths).
Each storage system sees all host adapters (use the storage show adapter command to display information about all or the specified adapter that is installed in a given slot).
Each storage system sees the all of the other storage systems (use the storage show initiator command to see a list of the initiator systems in the community).

Using software-based disk ownership

SharedStorage uses software-based disk ownership. For information on how to manage disks using software-based ownership, see Software-based disk ownership.

You assign disks in a community using the same command as you do for single or clustered storage systems under most circumstances. However, there are a few exceptions:

You can unassign disk ownership of a disk that is owned by a storage system by assigning it as unowned, as shown in the following example:

shared_1> disk assign 0b.16 -s unowned -f

The result of this command is that the disk is returned to the unowned pool.

You can also assign ownership of spare disks from one storage system to another, as shown in the following example:

shared_1> disk assign 0b.17 -o shared_2 -f

If there is a communication problem between the two storage systems, you will see warnings about "rescan messages".

Managing disks with SharedStorage

If you use the Data ONTAP command-line interface (CLI), you should assign disks and spares to each storage system and leave the rest in a large unowned pool. Assign disks from the unowned pool when you want to

Increase the size of an aggregate or a traditional volume if you are using the vFiler no-copy migration feature
Add a new aggregate or a traditional volume if you are using the vFiler no-copy migration feature
Replace a failed disk

If you use the FilerView or DataFabric® Manager graphical user interface, which do not recognize an unowned disk pool, you should assign all of the disks as spares to one storage system. This makes it easier to reassign disks for the tasks listed above.

Note
If you always use volumes of the same size, you can reassign all volumes and all vFiler units, and migrate a vFiler unit to the required storage system, when necessary.

Managing spare disks

If Data ONTAP needs a spare disk to replace a failed disk, it selects one that is assigned to that storage system. You should assign as many spares as possible to storage systems that are experiencing a higher disk failure rate. If necessary, you can assign more disks from the unowned pool of spare disks.

Allocating disks

If a storage system needs more storage, use the disk assign command to reassign spare disks to that storage system. The newly reassigned disks are then added to the traditional volume.

Note
You cannot assign disks to qtrees or FlexVol volumes.

Displaying information about disks

To see information about the disks owned by one storage system, complete the following step.

Step	Action
1	Enter the following command: `shared_1>` `disk show` DISK OWNER POOL SERIAL NUMBER ------ ----------- -------- ------ -------------- 9b.19 shared_1 (0050408412) Pool0 3HZ6RA1B0000742SWC9 3a.22 shared_1 (0050408412) Pool0 3HZ6DGM000074310Z3A 2b.104 shared_1 (0050408412) Pool0 414W5505 2b.106 shared_1 (0050408412) Pool0 414X5475

Step

Action

Enter the following command:

shared_1> disk show

DISK     OWNER                       POOL      SERIAL NUMBER

------   -----------      --------  ------   --------------

9b.19    shared_1  (0050408412)      Pool0     3HZ6RA1B0000742SWC9

3a.22    shared_1  (0050408412)      Pool0     3HZ6DGM000074310Z3A

2b.104   shared_1  (0050408412)      Pool0     414W5505

2b.106   shared_1  (0050408412)      Pool0     414X5475

About initiators and targets

Each storage system can behave as an initiator or a target. The storage system behaves as an initiator when it reads and writes data to disks. The storage system behaves as a target when it communicates with disks and disk shelves to download firmware, share SES information with other storage systems or share information with an FC adapter card.

Displaying initiators

To display the initiators in a SharedStorage community, complete the following step.

Step	Action
1	Enter the following command: `shared_1>` `storage show initiators` HOSTNAME SYSTEM ID ---------------------- ----------------- shared_1 0050408412 (self) shared_2 0050408123 shared_3 0050408133 shared_4 0050408717

Step

Action

Enter the following command:

shared_1> storage show initiators

HOSTNAME                 SYSTEM ID

----------------------   -----------------

shared_1                0050408412 (self)

shared_2                0050408123

shared_3                0050408133

shared_4                0050408717

To display the all of the initiators in the loop, complete the following step.

Step	Action
1	Enter the following command: `shared_1>` `fcadmin device_map` Loop Map for channel 3b: Translated Map: Port Count 73 0 7 16 17 18 19 20 21 22 23 24 25 26 27 28 29 32 33 34 35 36 37 38 39 40 41 42 43 44 45 48 49 50 51 52 53 54 55 56 57 58 59 60 61 80 81 82 83 84 85 86 87 88 89 90 91 92 93 96 97 98 99 100 101 102 103 104 105 106 107 108 109 1 2 Shelf mapping: Shelf 1: 29 28 27 26 25 24 23 22 21 20 19 18 17 16 Shelf 2: 45 44 43 42 41 40 39 38 37 36 35 34 33 32 Shelf 3: 61 60 59 58 57 56 55 54 53 52 51 50 49 48 Shelf 5: 93 92 91 90 89 88 87 86 85 84 83 82 81 80 Shelf 6: 109 108 107 106 105 104 103 102 101 100 99 98 97 96 Initiators on this looop: 0 (self) 1 (shared_2) 7 (shared_3) 2 (shared_4)

Step

Action

Enter the following command:

shared_1> fcadmin device_map

Loop Map for channel 3b:

Translated Map: Port Count 73

   0   7  16  17  18  19  20  21  22  23  24  25  26  27  28  29

  32  33  34  35  36  37  38  39  40  41  42  43  44  45  48  49

  50  51  52  53  54  55  56  57  58  59  60  61  80  81  82  83

  84  85  86  87  88  89  90  91  92  93  96  97  98  99 100 101

 102 103 104 105 106 107 108 109   1   2



Shelf mapping:

 Shelf 1:   29  28  27  26  25  24  23  22  21  20  19  18  17  16

 Shelf 2:   45  44  43  42  41  40  39  38  37  36  35  34  33  32

 Shelf 3:   61  60  59  58  57  56  55  54  53  52  51  50  49  48

 Shelf 5:   93  92  91  90  89  88  87  86  85  84  83  82  81  80

 Shelf 6:  109 108 107 106 105 104 103 102 101 100  99  98  97  96



Initiators on this looop:

   0 (self) 1 (shared_2) 7 (shared_3) 2 (shared_4)

vFiler no-copy migration software

The vFiler no-copy migration software supports NFS (non-disruptive) and CIFS. If you want to use the vFiler no-copy migration software, you are restricted to creating only traditional volumes and you must have the following licenses installed on your storage systems:

SnapMover
MultiStore

There are a few limitations with the vFiler migrate feature:

Root volumes in vFiler units cannot be migrated.
vFiler functionality is not supported for iSCSI or FCP.
When you move a volume to another storage system, VSM, QSM, and NDMP relationships must be re-established on the new storage system.
You can only move entire vFiler units. Use a one-to-one ratio for mapping traditional volumes to vFiler units.

With vFiler no-copy migration software installed, you can perform the following tasks:

Perform non-disruptive maintenance
You can isolate storage systems and disks, take them offline, perform maintenance and bring them back online without taking a loop out of service.

The SharedStorage hubs allow for multiple paths to the storage, which allow for hot swappable ESH controller modules and the ability to take one path to the storage offline, even in a CFO pair.

With vFiler no-copy migration functionality, you can migrate a traditional volume from one storage system to another, thereby isolating the first storage system to perform system maintenance while the target storage system continues to serve data.
Coordinate disk and shelf firmware downloads
SharedStorage technology ensures there is no disruption of service to all of the storage systems in the community when disk or disk shelf firmware is being downloaded to any disk or disk shelf.
Balance workloads amongst the storage systems using vFiler no-copy migration

Balancing workloads amongst the community

You can balance workloads amongst the storage systems in the community by migrating traditional volumes that are associated with vFiler units. If one storage system in the community is CPU-bound with the workload from one vFiler unit, you can migrate that unit to another storage system within seconds using the no-copy migration feature of SnapMover. For example, if you have four storage systems and one has a heavier load than the other three, use SnapMover to re-assign disks from CPU-bound head to another storage system. First you create a vFiler unit. Then you move its IP address by migrating from the overburdened storage system to an under-utilized storage system. The disks containing the volume change ownership from the overburdened storage system to the new one. As a result, you can balance traditional volumes across multiple storage systems in the community. For more information, see the MultiStore Management Guide.