Library Table of Contents Index Previous Page (Alt-p) Next Page (Alt-n) Comments

Disk and FC-AL messages from the Storage Health Monitor

About Storage Health Monitor messages  Top

The Storage Health Monitor examines FC-AL, disk shelf, and disk events and provides two kinds of syslog messages about the events: urgent and nonurgent. Each message has a corresponding message that is sent through autosupport and is available through SNMP. One set of message strings is provided for each event. There is also an informational message.

Responding to a predicted failure event message  Top

A syslog message about a predicted failure event is an urgent message and should be acted on immediately.

Syslog message

shm: disk has reported a predicted failure (PFA) 
event: disk XX, serial_number XXXX 

Autosupport and SNMP message

DISK_FAILURE_PREDICTED!!!

The Autosupport priority is alert.

Description of the problem

The disk has reported a Predicted Failure (PFA) Event. The disk's internal error processing and logging algorithm computation results are exceeding an internally set threshold. The disk is bound to fail in a matter of hours.

Action

If you experience a predicted failure event, complete the following step.

Step

Action

1

Replace the disk, as described in Chapter 5, Disk Management.

Responding to an FC-AL link failure message  Top

A syslog message about an FC-AL link failure is an urgent message and should be acted on immediately.

Syslog message

shm: link failure detected, upstream from disk: id 
XX, serial_number XXXXX

Autosupport and SNMP message

FC-AL LINK_FAILURE

The Autosupport priority is err.

Description of the problem

A Fibre Channel disk or cable, if disks are in different disk shelves, might be malfunctioning and might have caused an open loop condition. This results in a loss of sync over 100 milliseconds for a downstream disk that reported it as a link failure.

Action

If you experience an FC-AL link failure, complete the following steps.

Step

Action

Note
The following steps can take several hours to complete.

1

Use the halt command to halt the filer.

2

Check the upstream neighbor device of this disk and the cable between the two if they are on different disk shelves.

3

Take out two adjacent disks and any connecting cable.

4

Replace the cable, if any, to complete the loop.

5

Bring the filer up.

6

Use the disk scrub command.

Result:

If this link is bad, the syslog message reappears.

7

Replace the cable.

8

Put in one disk.

9

Use the disk scrub command.

Result:

If this disk is bad, the syslog message reappears.

10

Put the other disk back in.

11

Use the disk scrub command.

Result:

If this disk is bad, the syslog message reappears.

12

Replace the malfunctioning disk.

Responding to an "FC-AL unreliable" message  Top

An "FC-AL unreliable" message is a nonurgent message and does not require immediate action. However, you should take the actions listed in the Action column of the following table. If you need assistance in performing the actions, contact Network Appliance Technical Support.

Syslog message

shm: possible link errors on disk: id XX, serial 
number XXXXX. 

Autosupport and SNMP message

FC-AL UNRELIABLE

The Autosupport priority is warning.

Description of the problem

One of a group of four Fibre Channel disks in a disk shelf or any connecting cable might be malfunctioning.

This results in a large number of invalid_CRC frames and data underruns on the loop. The invalid_CRC and underrun count has crossed its set threshold several times.

Action

If you receive an "FC-AL unreliable message," complete the following steps.

Step

Action

Note
The following steps can take several hours to complete.

1

Use the halt command to halt the filer.

2

Take out the four disks in the group and any connecting cables that are reported as problems in the syslog messages.

3

Replace the cable, if any, to complete the loop.

4

Bring the filer up.

5

Use the disk scrub command.

Result:

If this link is bad, the syslog message reappears.

6

Put in one disk.

7

Use the disk scrub command.

Result:

If this disk is bad, the syslog message reappears.

8

Put the disks and cables back in, one at a time, performing Steps 3 through 7 to find the malfunctioning item.

9

Replace the malfunctioning item.

Responding to a disk degraded message  Top

A disk degraded message is a nonurgent message and does not require immediate action. However, you should take the action listed in the Action column of the following table. If you need assistance in performing the action, contact Network Appliance Technical Support.

Syslog message

shm: disk I/O Completion times too long: disk XX, 
serial number XXXXX. 

Autosupport and SNMP message

DISK_DEGRADED-I/O

The Autosupport priority is warning.

Description of the problem

Either the disk is old and slow, or it is internally recovering errors, thereby taking too long to complete an I/O. The message also indicates that there are too many I/O timeouts and retries on a disk. The disk might also be returning a Command Aborted status quite often. All these can result in a low data-throughput rate for that disk, and lowered overall system performance.

Action

If you receive a disk degraded message, complete the following step.

Step

Action

1

The disk is failure-prone; you should replace it as described in Chapter 5, Disk Management.

Responding to a disk recovered errors message  Top

A disk recovered errors message is a nonurgent message and does not require immediate action. However, you should take the action listed in the Action column of the following table. If you need assistance in performing the actions, contact Network Appliance Technical Support.

Syslog message

shm: disk returns excessive recovered errors: 
disk XX, serial number XXXXX. 

Autosupport and SNMP message

DISK_RECOVERED_ERRORS

The Autosupport priority is warning.

Description of the problem

Either the disk has found medium or hardware errors (unrecovered errors), or it has internally recovered numerous errors. The disk might also be returning a Command Aborted status. Quite often the errors returned exceed by many times the disk-vendor-specified bit error rate.

Action

If you receive a disk degraded message, complete the following step.

Step

Action

1

The disk is failure-prone; you should replace it as described in Chapter 5, Disk Management.

Responding to a loop intermittent instability message  Top

A loop intermittent instability message is an informational message and does not require any action.

Syslog message

shm: intermittent instability on the loop that is 
attached to Fibre channel adapter: id XXX, name XXXXX. NOTE: no 
immediate action needed.

Autosupport and SNMP message

None

Description of the problem

A Fibre Channel adapter, attached disk shelf, disk, cable, or connectors might cause instability on the FC-AL loop. It results in I/O completion rates that are below a set threshold.

Library Table of Contents Index Previous Page (Alt-p) Next Page (Alt-n) Comments