Cisco Fault Management User Manual
Cisco Active Network Abstraction Fault Management User Guide Version 3.6 Service Pack 1
THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO
CHANGE WITHOUT NOTICE. ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS
MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY
PRODUCTS.
Americas Headquarters
Cisco Systems, Inc.
170 West Tasman Drive
San Jose, CA 95134-1706
USA
http://www.cisco.com
Tel: 408 526-4000
800 553-NETS (6387)
Fax: 408 527-0883
Text Part Number: OL-14284-01
THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT
SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE
OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.
The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB’s public
domain version of the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.
NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS” WITH
ALL FAULTS. CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT
LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF
DEALING, USAGE, OR TRADE PRACTICE.
IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING,
WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO
OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
CCVP, the Cisco logo, and the Cisco Square Bridge logo are trademarks of Cisco Systems, Inc.; Changing the Way We Work, Live, Play, and Learn is a service mark of Cisco Systems,
Inc.; and Access Registrar, Aironet, BPX, Catalyst, CCDA, CCDP, CCIE, CCIP, CCNA, CCNP, CCSP, Cisco, the Cisco Certified Internetwork Expert logo, Cisco IOS, Cisco
Press,
Cisco Systems, Cisco Systems Capital, the Cisco Systems logo, Cisco Unity, Enterprise/Solver, EtherChannel, EtherFast, EtherSwitch, Fast Step, Follow Me Browsing,
FormShare, GigaDrive, HomeLink, Internet Quotient, IOS, iPhone, IP/TV, iQ Expertise, the iQ logo, iQ Net Readiness Scorecard, iQuick Study, LightStream, Linksys,
MeetingPlace, MGX, Networking Academy, Network Registrar, PIX, ProConnect, ScriptShare, SMARTnet, StackWise, The Fastest Way to Increase Your Internet Quotient, and
TransPath are registered trademarks of Cisco Systems, Inc. and/or its affiliates in the United States and certain other countries.
All other trademarks mentioned in this document or Website are the property of their respective owners. The use of the word partner does not imply a partnership relationship
between Cisco and any other company. (0708R)
Any Internet Protocol (IP) addresses used in this document are not intended to be actual addresses. Any examples, command display output, and figures included in the
document are shown for illustrative purposes only. Any use of actual IP addresses in illustrative content is unintentional and coincidental.
Cisco Active Network Abstraction Fault Management User Guide, Version 3.6 Service Pack 1
© 1999-2007 Cisco Systems, Inc. All rights reserved.
CONTENTS
About This Guide vii
Obtaining Documentation, Obtaining Support, and Security Guidelines vii
CHAPTER
CHAPTER
1 Fault Management Overview 1-1
Managing Events 1-1
Basic Concepts and Terms 1-2
Alarm 1-2
Event 1-3
Event Sequence 1-3
Repeating Event Sequence 1-4
Flapping Events 1-4
Correlation By Root Cause 1-5
Ticket 1-5
Sequence Association and Root Cause Analysis 1-6
Severity Propagation 1-6
Event Processing Overview 1-7
2 Fault Detection and Isolation 2-1
Unreachable Network Elements 2-1
Sources of Alarms On a Device 2-3
Alarm Integrity 2-3
CHAPTER
OL-14284-01
Integrity Service 2-3
3 Cisco ANA Event Correlation and Suppression 3-1
Event Suppression 3-1
Root-Cause Correlation Process 3-2
Root-Cause Alarms 3-3
Correlation Flows 3-3
Correlation by Key 3-3
Correlation by Flow 3-3
DC Model Correlation Cache 3-4
Using Weights 3-4
Cisco Active Network Abstraction Fault Management User Guide, Version 3.6 Service Pack 1
iii
Contents
Correlating TCA 3-4
CHAPTER
4 Advanced Correlation Scenarios 4-1
Device Unreachable Alarm 4-1
Connectivity Test 4-1
Device Fault Identification 4-2
Device Unreachable Example 1 4-2
Device Unreachable Example 2 4-2
IP Interface Failure Scenarios 4-3
IP Interface Status Down Alarm 4-3
Correlation of Syslogs and Traps 4-4
All IP Interfaces Down Alarm 4-5
IP Interface Failure Examples 4-5
Interface Example 1 4-6
Interface Example 2 4-6
Interface Example 3 4-7
Interface Example 4 4-7
Interface Example 5 4-8
ATM Examples 4-9
Ethernet, Fast Ethernet, Giga Ethernet Examples 4-9
Interface Example 6 4-9
Interface Example 7 4-9
Interface Registry Parameters 4-10
ip interface status down Parameters 4-10
All ip interfaces down Parameters 4-10
iv
Multi Route Correlation 4-11
Multi Route Correlation Example 1 4-11
Multi Route Correlation Example 2 4-11
Multi Route Correlation Example 3 4-12
Multi Route Correlation Example 4 4-13
Generic Routing Encapsulation (GRE) Tunnel Down/Up 4-13
GRE Tunnel Down/Up Alarm 4-13
GRE Tunnel Down Correlation Example 1 4-14
GRE Tunnel Down Correlation Example 2 4-15
BGP Process Down Alarm 4-17
MPLS Interface Removed Alarm 4-17
LDP Neighbor Down Alarm 4-17
Cisco Active Network Abstraction Fault Management User Guide, Version 3.6 Service Pack 1
OL-14284-01
Contents
CHAPTER
CHAPTER
CHAPTER
5 Correlation Over Unmanaged Segments 5-1
Cloud VNE 5-1
Types of Unmanaged Networks Supported 5-1
Fault Correlation Across the Frame Relay or ATM or Ethernet Cloud 5-2
Cloud Problem Alarm 5-3
Cloud Correlation Example 5-3
6 Event and Alarm Configuration Parameters 6-1
Alarm Type Definition 6-1
Event (Sub-Type) Configuration Parameters 6-2
General Event Parameters 6-2
Root Cause Configuration Parameters 6-2
Correlation Configuration Parameters 6-3
Network Correlation Parameters 6-3
Flapping Event Definitions Parameters 6-4
System Correlation Configuration Parameters 6-4
7 Impact Analysis 7-1
APPENDIX
Impact Analysis Options 7-1
Impact Report Structure 7-2
Affected Severities 7-2
Impact Analysis GUI 7-3
Affected Parties Tab 7-3
Viewing a Detailed Report For the Affected Pair 7-4
Disabling Impact Analysis 7-6
Accumulating Affected Parties 7-6
Accumulating the Affected Parties In an Alarm 7-7
Accumulating the Affected Parties In the Correlation Tree 7-7
Updating Affected Severity Over Time 7-7
A Supported Service Alarms A-1
Shelf Out A-4
Rx Dormant A-5
Tx Dormant A-5
Link Over Utilized A-5
OL-14284-01
Cisco Active Network Abstraction Fault Management User Guide, Version 3.6 Service Pack 1
v
Contents
APPENDIX
B Event and Alarm Correlation Flow B-1
Software Function Architecture B-2
Event Correlation Flow B-3
Event Creation (VNE level) B-3
Event Correlation B-3
Local Correlation (Event Correlator) B-3
Network Correlation (Event Correlator, Flow) B-3
Correlation Logic (Event Correlator) B-4
Alarm Sending (Event Correlator) B-4
Post-Correlation Rule (Event Correlator) B-4
vi
Cisco Active Network Abstraction Fault Management User Guide, Version 3.6 Service Pack 1
OL-14284-01
About This Guide
This guide includes the following chapters:
• Chapter 1, “Fault Management Overview”—Describes how to manage events, and introduces some
of the key concepts of Cisco ANA alarm management.
• Chapter 2, “Fault Detection and Isolation”—Describes unreachable network elements and the
sources of alarms on devices. In addition, it describes alarm integrity and the integrity service
• Chapter 3, “Cisco ANA Event Correlation and Suppression”—Describes how Cisco ANA performs
correlation logic decisions.
• Chapter 4, “Advanced Correlation Scenarios”—Describes specific alarms which use advanced
correlation logic on top of the root cause analysis flow.
• Chapter 5, “Correlation Over Unmanaged Segments”—Describes how Cisco ANA performs
correlation decisions over unmanaged segments.
• Chapter 6, “Event and Alarm Configuration Parameters”—Describes the details of various
configurable alarm parameters.
• Chapter 7, “Impact Analysis”—Describes the impact analysis functionality available in Cisco ANA.
• Appendix A, “Supported Service Alarms”—Provides the list of service alarms that are supported in
Cisco ANA.
• Appendix B, “Event and Alarm Correlation Flow”—Describes in detail the flow of alarms and
events during the correlation process.
Obtaining Documentation, Obtaining Support, and Security Guidelines
For information on obtaining documentation, obtaining support, providing documentation feedback,
security guidelines, and also recommended aliases and general Cisco documents, see the monthly
What’s New in Cisco Product Documentation, which also lists all new and revised Cisco technical
documentation, at:
http://www.cisco.com/en/US/docs/general/whatsnew/whatsnew.html
Cisco Active Network Abstraction Fault Management User Guide, Version 3.6 Service Pack 1
OL-14284-01
vii
Obtaining Documentation, Obtaining Support, and Security Guidelines
About This Guide
viii
Cisco Active Network Abstraction Fault Management User Guide, Version 3.6 Service Pack 1
OL-14284-01
Fault Management Overview
This chapter describes the challenge of managing an overabundance of events, and introduces some of
the key concepts of Cisco ANA alarm management.
• Managing Events—Describes how to manage events effectively.
• Basic Concepts and Terms—Describes the basic concepts and terms used throughout this guide.
• Severity Propagation—Describes the concept of severity, and how severity is propagated.
• Event Processing Overview—Describes the process for identifying and processing raw events.
Managing Events
The challenge of dealing effectively with events and alarms is to know how to understand and efficiently
process and organize bulks of raw events that may be generated as a result of single root cause events.
CHA PTER
1
OL-14284-01
Cisco Active Network Abstraction Fault Management User Guide, Version 3.6 Service Pack 1
1-1
Basic Concepts and Terms
Figure 1-1 Event Flood
Chapter 1 Fault Management Overview
Lost
Connectivity
Trap: DLSw
Peer Down
Syslog: HSRP
Standby -> Active
Syslog: FR
DLCI Down
! !
!
Unmanaged
Network
Syslog: Lost
BGP Neighbor
!
Ping: Device
Unreachable
IP Backbone
Trap: Link
Down
Syslog: Lost
OSPF neighbor
Ping: Device
Unreachable
!
Syslog
OSPF neighbor
!
!
!
: Lost
!
Syslog: LSP
Reroute
Syslog: LSP
Reroute
Trap: Link
Down
!
!
Syslog: Lost
!
BGP Neighbor
Syslog: Lost
OSPF neighbor
Trap: DLSw
Peer Down
Lost
Connectivity
Syslog: Lost
BGP Neighbor
Syslog: Lost
Neighbor
!
!
!
!
!
!
Meeting the event management challenge is done by correlating related events into a sequence that
represents the alarm lifecycle, and using the network dependency model to determine the causal
inter-relationship between alarms.
Cisco ANA can be used for analyzing and managing faults using fault detection, isolation and
correlation. Once a fault is identified, the system uses the auto-discovered virtual network model to
perform fault inspection and correlation in order to determine the root cause of the fault and, if
applicable, to perform service impact analysis.
Basic Concepts and Terms
Alarm
An alarm represents a scenario which involves a fault occurring in the network or management system.
Alarms represent the complete fault lifecycle, from the time that the alarm is opened (when the fault is
first detected) until it is closed and acknowledged. Examples of alarms include:
• Link down
• Device unreachable
154391
1-2
Cisco Active Network Abstraction Fault Management User Guide, Version 3.6 Service Pack 1
OL-14284-01
Chapter 1 Fault Management Overview
• Card out
• An alarm is composed of a sequence of events, each representing a specific point in the alarm’s
lifecycle.
Event
An event is an indication of a distinct occurrence that occurred at a specific point in time. Events are
derived from incoming traps and notifications, and from detected status changes. Examples of events
include:
• Port status change.
• Connectivity loss between routing protocol processes on peer routers (for example BGP neighbor
loss).
• Device reset.
• Device becoming reachable by the management station.
• User acknowledgement of an alarm.
Events are written to the Cisco ANA database once and never change.
The collected events are displayed in Cisco ANA EventVision. Refer to the Cisco Active Netowrk
Abstraction EventVision User Guide for more information.
Basic Concepts and Terms
Event Sequence
An event sequence is the set of related events which comprises a single alarm. For example,
link down > ack > link up.
Figure 1-2 Event Sequence Example
Typically, a complete event sequence includes three mandatory events:
• Alarm open (in this example a link-down event).
• Alarm clear (in this example a link-up event).
• Alarm acknowledge.
OL-14284-01
Optionally, there can be any number of alarm change events which can be triggered by new severity
events, affected services update events, and so on.
Cisco Active Network Abstraction Fault Management User Guide, Version 3.6 Service Pack 1
1-3
Basic Concepts and Terms
Note The event types that will belong to each sequence can be configured in the system registry.
An event sequence can consist of a single event (for example, “device reset”).
The set of events that should participate in Cisco ANA alarm processing can be configured in the system
registry.
Repeating Event Sequence
If a new opening event arrives within a configurable timeout after the clearing event of the same alarm,
the alarm is updatable, and a repeating event sequence is created, that is, the event is attached to the
existing sequence and updates its severity accordingly. If the new opening event occurs after the timeout,
it opens a new alarm (new event sequence).
Figure 1-3 Repeating Event Sequence
Chapter 1 Fault Management Overview
Flapping Events
If a series of events that are considered to be of a same sequence occur in the network in a certain
configurable time window a certain (configurable) amount of times, the virtual network element (VNE)
may (upon configuration) reduce further the number of events, and will issue a single event which will
be of type “event flapping”. Only when the alarm stabilizes and the event frequency is reduced, will
another update to the event sequence be issued as “event stopped flapping”. Another update will be
issued with the most up-to-date event state.
Cisco Active Network Abstraction Fault Management User Guide, Version 3.6 Service Pack 1
1-4
OL-14284-01
Chapter 1 Fault Management Overview
Figure 1-4 Flapping Event
Correlation By Root Cause
Basic Concepts and Terms
Root cause correlation is determined between alarms or event sequences. It represents a causal
relationship between an alarm and the consequent alarms that occurred because of it.
For example, a card-out alarm can be the root cause of several link-down alarms, which in turn can be
the root cause of multiple route-lost and device unreachable alarms, and so on. A consequent alarm can
serve as the root cause of other consequent alarms.
Figure 1-5 Root Cause Correlation Hierarchy Example
Ticket
OL-14284-01
A ticket represents the complete alarm correlation tree of a specific fault scenario. It can be also
identified by the topmost or “root of all roots” alarm. Both Cisco ANA NetworkVision and Cisco ANA
EventVision display tickets and allow drilling down to view the consequent alarm hierarchy.
Cisco Active Network Abstraction Fault Management User Guide, Version 3.6 Service Pack 1
1-5
Severity Propagation
From an operator’s point of view, the managed entity is always a complete ticket. Operations such as
Acknowledge, Force-clear or Remove are always applied to the whole ticket. The ticket also assumes an
overall, propagated severity.
Sequence Association and Root Cause Analysis
There are two different types of relationships in Cisco ANA alarm management:
• Sequence Association—The association between events, which creates the event sequences and
alarms.
• Root Cause Analysis—The association between alarms (event sequences) which represents the root
cause relationship.
The following figure shows how both types of relationship are implemented in the ticket hierarchy:
Figure 1-6 Sequence Association vs. Root Cause Analysis
Chapter 1 Fault Management Overview
In the above figure, the alarms are correlated into a hierarchy according to root cause. Within each alarm
is its respective event sequence representing the lifecycle of the alarm.
Severity Propagation
Each event has an assigned severity (user-configurable). For example, a link-up event may be assigned
critical severity, while its corresponding link-up event will have normal severity.
Cisco Active Network Abstraction Fault Management User Guide, Version 3.6 Service Pack 1
1-6
OL-14284-01
Chapter 1 Fault Management Overview
The propagated severity of the alarm (the whole event sequence) is always determined by the last event
in the sequence. In the above example, when the link-down alarm is open it will have critical severity;
when it clears it moves to normal severity. An exception to this rule is the informational event (severity
level of info) such as user acknowledge event, which does not change the propagated severity of the
sequence (the alarm).
Each ticket assumes the propagated severity of the alarm with the topmost severity, within all the alarms
in the correlation hierarchy at any level.
Note Each alarm does not assume the propagated severity of the correlated alarms beneath it. Each alarm
assumes its severity only from its internal event sequence, as described above, while the ticket assumes
the highest severity among all the alarms in the correlation tree.
Event Processing Overview
Cisco ANA provides a customizable framework for identifying and processing raw events. The raw
events are collected into the Event Manager, forwarded to their respective VNE, and then processed as
follows:
Event Processing Overview
Step 1 The event data is parsed to determine its source, type, and alarm-handling behavior.
Step 2 If the event type is configured to try and correlate, the VNE attempts to find a compliant cause alarm.
This is done in the VNE fabric.
Step 3 The event fields are looked up and completed.
Step 4 The event is sent to the Cisco ANA gateway, where:
• The event is written to the event database.
• If the event belongs to an alarm, it is attached to its respective event sequence and correlated to the
respective root-cause alarm within the ticket, or a new sequence and new ticket is opened.
• If the event is marked as ticketable, and it did not correlate to any other alarm, a new ticket will be
opened where the alarm that triggered the ticket will be the root cause of any alarms in the
correlation tree.
OL-14284-01
Cisco Active Network Abstraction Fault Management User Guide, Version 3.6 Service Pack 1
1-7
Event Processing Overview
Chapter 1 Fault Management Overview
1-8
Cisco Active Network Abstraction Fault Management User Guide, Version 3.6 Service Pack 1
OL-14284-01
Fault Detection and Isolation
This chapter describes unreachable network elements and the sources of alarms on devices. In addition,
it describes alarm integrity and the integrity service:
• Unreachable Network Elements—Describes how the various VNEs use reachability to check
connectivity with the NEs.
• Sources of Alarms On a Device—Describes the four basic alarm sources that indicate problems in
the network.
• Alarm Integrity—Describes what happens when a VNE with associated open alarms shuts down.
• Integrity Service—Describes the integrity service tests that run on the gateway and/or the units.
Unreachable Network Elements
Reachability used by the VNEs (checks the reachability between the VNEs and NEs) depends on the
configuration of the VNE, and involves multiple connectivity tests, using SNMP, Telnet/SSH and/or
ICMP, as appropriate.
The table describes the various situations below when a NE fails to respond to the protocols:
CHA PTER
2
Table 2-1 Unreachable Network Elements
VNE Type Checks reachability using When the NE fails to respond When the NE is reachable
ICMP VNE ICMP only. During the ICMP
test the unit pings the NE
every configured interval.
Cisco Active Network Abstraction Fault Management User Guide, Version 3.6 Service Pack 1
OL-14284-01
ICMP ping is suspended, and a VNE
Unreachable alarm is sent to the
Cisco ANA Gateway. Only the
reachability tests are executed thereafter to
detect when the device is reachable again.
ICMP ping is restarted, and the
alarm is cleared.
2-1
Chapter 2 Fault Detection and Isolation
Unreachable Network Elements
Table 2-1 Unreachable Network Elements (continued)
VNE Type Checks reachability using When the NE fails to respond When the NE is reachable
Generic
VNE
Full VNE
• SNMP only (default).
During the SNMP test the
unit’s “
SNMP get” the
sysoid of the NE and
expects to receive a
response
or
• SNMP only (default),
and adding an ICMP test.
• SNMP only (default).
During the SNMP
reachability test, the
VNE polls the device’s
SysOID MIB using a
standard “
SNMP Get”
command, and expects to
receive a response
or
• SNMP only (default),
General polling is suspended, and a VNE
Unreachable alarm is sent to the
Cisco ANA Gateway. Only the
reachability tests are executed thereafter to
detect when the device is reachable again.
If more than one protocol is used, it is
enough for one of them to become
unreachable in order to generate the alarm.
The alarm is generic to all the protocols.
General polling is suspended, and a VNE
Unreachable alarm is sent to the
Cisco ANA Gateway. Only the
reachability tests are executed thereafter to
detect when the device is reachable again.
If more than one protocol is used, it is
enough for one of them to become
unreachable in order to generate the alarm.
The alarm is generic to all the protocols.
• General polling is restarted.
• The first time the VNE is
started, all the commands are
submitted to the queue, and the
collector initiates an
immediate session with the
NE. The commands are sent to
the NE in a serial fashion.
• The alarm is cleared.
• The first time the VNE is
started, all the commands are
submitted to the queue and the
collector initiates an
immediate session with the
NE. The commands are sent to
the NE in a serial fashion.
• The alarm is cleared.
and adding ICMP and
Teln e t. Durin g the Tel n et
test the unit sends
"
Enter" via the open
session and expects to get
a prompt back.
2-2
Each of these scenarios have two possible settings in the registry, namely:
• track reachability (true/false). The default is true.
When this parameter is true reachabilty is tracked according to the specific protocol, for example,
ICMP, SNMP, Telnet, and so on.
When this parameter is false then the test is not performed.
• lazy reachability (true/false). The default is false. This parameters determines whether there is a
dedicated reachability command ‘in-charge’ of tracking reachability or whether reachability is
determined by the regular polled commands.
When this parameter is true reachability is based on polling, and a dedicated command not activated.
When this parameter is false a dedicated SNMP command is activated, and this test verifies the
response from a specific SNMP oid (sysoid is the default that can be changed).
Note Changes to the registry should only be carried out with the support of Cisco Professional Services.
Cisco Active Network Abstraction Fault Management User Guide, Version 3.6 Service Pack 1
OL-14284-01
Chapter 2 Fault Detection and Isolation
Sources of Alarms On a Device
The following basic sources of alarms exist in the system which indicate a problem in the network:
• Service Alarms—Alarms generated by the VNE as a result of polling (for example SNMP, Telnet).
Usually such alarms (for example link down, card out, device unreachable and so on) are configured
in such a way that they can become root cause alarms, according to the correlation algorithms.
Service alarms can also be generated by the gateway, for example. the vpn leak alarm.
• SNMP Traps—Traps sent by the network elements and captured by the Cisco ANA platform. The
platform supports SNMP v1, v2 and v3 traps. The traps are then forwarded to the specific VNEs for
further processing and correlation logic. In addition, reliable traps (inform commands) are
supported, when configured in the registry, where the VNE acknowledges that a trap was received.
• Syslogs—Syslog messages sent by the network elements and captured by the Cisco ANA platform.
The Syslogs are then forwarded to the specific VNEs for further processing and correlation logic.
• TCA—Cisco ANA can be used to set a TCA for soft properties. The TCA can be enabled to assign
a condition to the property which will trigger an alarm when violated. The alarm conditions could
be:
–
Equal or not equal to a target value.
–
Exceeding a defined value range (defined by maximum and minimum thresholds, including
hysteresis), for example CPU level of a device.
Sources of Alarms On a Device
–
• System Alarms—Alarms generated by the gateway and/or the units, for example, disk full, database
full, unit unreachable and so on. For more information see Integrity Service.
For information about TCAs see the Cisco Active Network Abstraction Customization User Guide.
Alarm Integrity
When the VNE shuts down while it still has open alarms associated with it, “fixing” events which occur
during the down period will be consolidated when the VNE is reloaded.
Integrity Service
The integrity service is an internal service that runs on the gateway and/or the units, which is responsible
for the stability of the system by running integrity tests in order to maintain the database and eliminate
clutter in the system. In order to prevent the session from stopping, the integrity service tests are run on
a different thread in a separate directory called integrity.
The service integrity tests are run:
• Manually—The integrity service tests are accessed as part of the Cisco ANA Shell management
services, and they can be accessed by telneting the gateway.
Exceeding a defined rate (calculated across time), for example bandwidth or utilization rate of
a link.
OL-14284-01
To run a test, the user should cd to the integrity dir, and then enter
name. The user can pass parameters to the tests using Cisco ANA Shell.
• Automatically—The integrity service tests are scheduled as crontab commands, to run specific tests
at specific intervals. By default the integrity service tests run automatically every 12 hours.
Cisco Active Network Abstraction Fault Management User Guide, Version 3.6 Service Pack 1
executeTest followed by the test
2-3
Integrity Service
Note Changes to the registry should only be carried out with the support of Cisco Professional Services.
Chapter 2 Fault Detection and Isolation
For example, this line in crontab runs the file every_12_hours.cmd at 11:00AM and 11:00PM:
0 11,23 * * * local/cron/every_12_hours.cmd > /dev/null 2>&1
The integrity service tests can be defined inside the cmd file, for example:
echo “`date '+%d/%m/%y %H:%M:%S -'` running integrity.executeTest alarm”
cd ~/Main ; ./mc.csh localhost 8011 integrity.executeTest alarm >& /dev/null
The first line prompts the user when a test starts to run, the next line runs the test.
The integrity service test parameters are defined in the registry. The registry entries responsible for the
integrity service can be found at:
mmvm/agents/integrity
The integrity service tests include, for example, the following:
• Alarm—Deletes cleared alarms if the alarm count is above the defined threshold.
• businessObject—Checks for invalid OIDs in business objects.
• Capacity—Checks the disk space capacity.
• archiveLogs—Deletes Oracle logs.
• tablespace—Checks that there is enough disk space for tablespace growth.
• workflowEngine—Deletes all complete workflows that started before a configured period of time.
2-4
Cisco Active Network Abstraction Fault Management User Guide, Version 3.6 Service Pack 1
OL-14284-01
Cisco ANA Event Correlation and Suppression
This chapter describes how Cisco ANA performs correlation logic decisions:
• Event Suppression—Describes enabling or disabling port-down, port-up, link-down and link-up
alarms on a selected port.
• Root-Cause Correlation Process—Describes the root-cause correlation concept.
• Root-Cause Alarms—Describes the root-cause alarm and weights concepts.
• Correlation Flows—Describes correlation by flow and correlation by key. In addition, it describes
the DC model correlation cache.
Event Suppression
The user can enable or disable the port-down, port-up, link-down, and link-up alarms on a selected port.
By default, alarms are enabled on all ports except for xDSL. When the alarms are disabled on a port, no
alarms will be generated for the port, and they will not be displayed in the ticket pane. Using the Registry
Editor advanced tool, it is possible to enable or disable service alarms on network entities other than
ports, such as the MPBGP (for enabling or disabling BGP neighbor down events), or the MPLS TE
Tunnel (for TE-Tunnel down service alarm). It is also possible to enable or disable alarm specific types
without regard to a specific network entity.
By default, port-down alarms are suppressed on xDSL ports. Cisco ANA supports selectively enabling
sending of port-down alarms on xDSL ports. This can be done by:
CHA PTER
3
OL-14284-01
• Using a command available in the GUI, right-click on the port in the inventory, select Enable
Sending Alarms.
or
• Setting a flag in the registry under the OID of the port. Changes to the registry should only be carried
out with the support of Cisco Professional Services.
Refer to the Cisco Active Network Abstraction NetworkVision User Guide for information about
disabling or enabling a port alarm.
Events can also be filtered according to their DC type source, for example, all the events that come from
any ATM DC can be filtered by configuring the registry. The following alarm under DC types is filtered
by default:
• VRF—duplicate ip on vpn
Cisco Active Network Abstraction Fault Management User Guide, Version 3.6 Service Pack 1
3-1
Loading...