This section covers the following topics:
The filer uses RAID Level 4 to ensure data integrity even when one of the disks fails. The file system design, together with the support for RAID, optimizes filer performance and enables you to incrementally expand the filer's disk storage capacity.
In a RAID group, different disks have different functions. Most of the disks in the RAID group are data disks. One disk is the parity disk, which enables the filer to recover the data on a data disk if one fails.
The filer supports multiple RAID groups. The factory default filer configuration contains one RAID group. The filer supports up to 32 RAID groups. Each RAID group belongs to only one volume; you cannot assign more than one volume to the same RAID group.
Spare disks are used by RAID groups as needed. They do not have to be in the same disk shelf to be available to a RAID group. The filer automatically assigns disks to RAID groups and creates new RAID groups as each RAID group is filled with its maximum number of disks.
The following characteristics apply to RAID group size:
Occasionally, you might encounter situations in which you want to specify a RAID group size other than the default. For example, you might want to configure a filer with smaller RAID groups for the following reasons:
Conversely, configuring with larger (therefore, fewer) RAID groups in a filer uses fewer disks for parity, leaving more disks available for data storage.
In each RAID group, the filer assigns the role of parity disk to the largest disk in the RAID group. After a data disk failure, the filer uses the parity disk in conjunction with the other data disks to reconstruct the failed disk's data and optionally write it to a hot spare disk. The parity disk must be at least as large as the largest data disk.
For more information about how the filer recovers from disk failures by using the parity disk or hot spare disk, refer to "Handling disk failures."
You identify a disk by its address, which is listed in the HA.Disk_ID column of the output of the sysconfig -r command. In this output listing, HA refers to the host adapter and Disk_ID refers to the disk ID number.
You use the disk address to
With SCSI disks, the disk address is a combination of the disk's SCSI adapter number and the disk's SCSI ID number on the adapter. For example, disks attached to SCSI adapter 0 are numbered 0.0 through 0.6, disks attached to SCSI adapter 9a are numbered 9a.0 through 9a.6, and so on.
With fibre channel disks, the disk address is a combination of the disk's adapter number and the disk's fibre channel loop ID. To create the fibre channel loop ID, multiply the shelf ID switch value by eight and add it to the bay number. For example, ha 8, shelf 1, disk 2 has disk ID 8.10.
For more information about locating specific drives on disk shelves, refer to the hardware guide for the disk shelf.
The filer uses the RAID disk scrubbing procedure to increase data availability. The filer scans each disk in the RAID group for media errors. If the filer finds media errors, it fixes them by reconstructing the data from parity and rewriting the data.
Disk scrubbing reduces the chance of a multiple disk failure, caused by a disk media error encountered while the system was running in degraded mode.
The filer only scans a RAID group when all the group's disks are operational. Although disk scrubbing slows the filer somewhat, network clients might not notice the change in the filer's performance because disk scrubbing starts automatically at 1:00 A.M. on Sunday, when most systems are lightly loaded.
By default, disk scrubbing is enabled. You might want to disable scrubbing if you have a recurring problem that scrubbing encounters.
For example, there might be an unrecoverable error on a disk that you cannot fix before the next disk scrub. The following commands disable and enable disk scrubbing:
options raid.scrub.enable off
options raid.scrub.enable on
You can also manually start and stop disk scrubbing regardless of the current value (On or Off) of the raid.scrub.enable option. Following are the commands for starting and stopping scrubbing manually:
disk scrub start
disk scrub stop
Messages from the disk scrubbing process are sent to the system error logging daemon.
Following are sample messages that can appear:
Inconsistent parity on volume volume_name, RAID group n, stripe #n.
Rewriting bad parity block on volume volume_name, RAID group n, stripe #n.
Note: An Inconsistent parity error message might indicate file system corruption. If you get such an error, contact Network Appliance Technical Support for assistance.
Rewriting bad parity block on volume volume_name, RAID group n, stripe #n
Rewriting bad block from parity on disk n, block n
Multiple bad blocks found on volume volume_name, RAID group n, stripe #n
The following sample messages appear after disk scrubbing is complete:
Scrub found n parity inconsistencies
Scrub found n media errors
Disk scrubbing finished...
In addition to data disks and parity disks, the filer supports zero or more hot spare disks. A hot spare disk is not part of any RAID group and does not contain file system data. After a disk failure, the filer automatically rebuilds data (or parity) onto a hot spare disk, which then replaces the failed disk in the RAID group. This procedure avoids a system shutdown and returns the system to full performance.
A hot spare disk cannot replace a failed disk that is larger than itself. If you use only one hot spare disk in a filer, the hot spare disk must be as large as the largest file system disk. If you have multiple hot spare disks installed, the system uses the smallest hot spare disk needed to replace the failed disk.
The filer enables you to hot swap disk drives. Hot swapping a disk drive means installing or removing it from the disk shelf while the filer is running, with minimal interruption to a file system. For example, you might want to hot swap a disk into a filer to replace a disk or to add a hot spare disk.
You can use disks of various sizes in the same filer. The following table shows a sample configuration.
The following sections discuss the guidelines for choosing the size of a disk to add to the filer or to use as a replacement.
When you replace a failed disk, the physical space of the new disk must be equal to or larger than the usable space of the replaced one. The Used column of the sysconfig -r output shows the amount of usable space on each disk.
However, if the physical space of the new disk is larger than the usable space of the old disk, the system can use no more space than was available on the old disk. For example, if you replace a 4-GB disk with a 9-GB disk, the extra 5 GB on the new disk is not available to the file system. Therefore, it is most cost-effective to replace a disk with a disk of the same size.
When you add to a file system a disk that is larger than the current parity disk, the new disk automatically becomes the parity disk. The disk used as the parity disk before the disk addition becomes a data disk.
A disk's usable space can be different from its physical space. The information here applies to data, parity, and hot spare disks:
The following table shows the usable disk space for each class of disk supported by the filer.
| 1-GB disks | 2-GB disks | 4-GB disks | 9-GB disks |
|---|---|---|---|
| 1,000 mb (2,048,000 blocks) | 2,000 mb (4,096,000 blocks) | 4,000 mb (8,192,000 blocks) | 9,000 MB (17,612,800 blocks) |
Although automatic disk right-sizing is not applied to existing disks in an upgraded system, it is applied to disks being added to the filer.
You can use sysconfig -r to compare the physical space and the usable space, and to determine whether disks are right-sized.
If one block on a data disk fails, the filer uses the parity disk in its RAID group to reconstruct the data on that block. The block is mapped to a new location on disk. If an entire data disk fails, the parity disk for that RAID group prevents any data loss and enables the filer to continue running.
Although the filer can continue to function with a failed disk, if it cannot reconstruct that failed disk on a hot spare, it automatically shuts down after 24 hours to encourage you to replace the failed disk. You can change the amount of time from 24 hours to another value using the raid.timeout option to the options command.
The effects of a disk failure on filer operation depend on whether the filer has a hot spare disk.
If the filer is not equipped with a hot spare disk, after a disk fails the filer enters a state called "degraded mode." In this state, the RAID feature enable the filer to continue to run without losing data (although the filer's performance is affected). Replace the failed disk as soon as possible because a second disk failure in the same RAID group causes the entire file system to be lost.
When a disk fails, the filer logs a warning message in the /etc/messages file and to the system console every hour, notifying you of the number of hours before the system shuts down.
The shutdown ensures that you notice the disk failure. You can restart the filer without fixing the disk, but the filer continues to shut itself off at the specified intervals until you repair the problem.
By default, the filer shuts down after 24 hours. You can change this time interval using the option raid.timeout command; the argument is the time, in hours, that the system runs before automatic shutdown.
The system shuts down after the specified period if it is running in degraded mode. A filer is in degraded mode if either of the following conditions exist:
If you reserve one or more disks as hot spare disks when you configure your filer, the filer also enters degraded mode after a disk failure. However, the filer immediately begins rebuilding the missing data in the background on the hot spare disk, with minimal interruption to file service.
The filer logs this activity in the /etc/messages file and does not automatically shut down. If you turn off the filer while it is in degraded mode, it stops data reconstruction. After you turn the filer back on, the filer restarts the data reconstruction process from the beginning.
Except for a loss in performance while data is rebuilt on the hot spare disk, the failure of a single disk is transparent to the user. The filer exits degraded mode and returns to normal operation after it finishes reconstructing the data.
Network Appliance recommends that you replace the failed disk with a new hot spare disk after the filer finishes reconstructing data. This way, the filer continues to have a hot spare disk that it can use in case another disk fails.
The sysconfig -r command displays which disk is reserved as the hot spare disk. In addition to disk failure and hot spare disk replacement activity, the /etc/messages file logs any failure in a periodic check of the hot spare disk.
You can control the speed of RAID data reconstruction by entering the following command:
options raid.reconstruct_speed speed
where speed is a number ranging from 1 (slowest) to 10 (fastest). Because RAID data reconstruction consumes CPU time, sometimes increasing the speed of data reconstruction slows the filer's network operations. The default speed is 4, which means approximately 40% of the CPU time is used for RAID data reconstruction.
When RAID data reconstruction is in progress, use the sysstat command to check the system load on the filer. If the load is light, increase the speed of RAID data reconstruction to maximize CPU utilization. For more information about the sysstat command, refer to "Displaying filer statistics."
Copyright © 1999, Network Appliance, Inc. All rights reserved.
Part No. 210-02166
File: disk1.htm
Last updated: 02/24/99 15:46:00
