The information contained in this manual is the property of Netcom Systems, Inc. and is furnished for use by
recipient only for the purpose stated in the Software License Agreement accompanying the user documentation.
Except as permitted by such License Agreement, no part of this publication may be reproduced, stored in a retrieval
system, or transmitted, in any form or by any means, without the prior written permission of Netcom Systems, Inc.
Information contained in the user documentation is subject to change without notice and does not represent a
commitment on the part of Netcom Systems, Inc. Netcom Systems, Inc. assumes no responsibility or liability for any
errors or inaccuracies that may appear in the user documentation.
Trademarks
SmartBits is a trademark of Netcom Systems, Inc.
Warranty
Netcom Systems, Inc. warrants to recipient that hardware which it supplies with this user documentation (“Product”)
will be free from significant defects in materials and workmanship for a period of twelve (12) months from the date of
delivery (the “Warranty Period”), under normal use and conditions.
Defective Product under warranty shall be, at Netcom Systems’ discretion, repaired or replaced or a credit issued to
recipient’s account for an amount equal to the price paid for such Product provided that: (a) such Product is returned to
Netcom Systems after first obtaining a return authorization number and shipping instructions, freight prepaid, to Netcom
Systems’ location in the United States; (b) recipient provide a written explanation of the defect claimed; and (c) the
claimed defect actually exists and was not caused by neglect, accident, misuse, improper installation, improper repair,
fire, flood, lightning, power surges, earthquake or alteration. Netcom Systems will ship repaired Product to recipient,
freight prepaid, within ten (10) working days after receipt of defective Product. Except as otherwise stated, any claim on
account of defective materials or for any other cause whatsoever will conclusively be deemed waived by recipient
unless written notice thereof is given to Netcom Systems within the Warranty Period. Product will be subject to
Netcom Systems’ standard tolerances for variations.
TO THE EXTENT PERMITTED BY APPLICABLE LAW, ALL IMPLIED WARRANTIES, INCLUDING BUT
NOT LIMITED TO IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT AND
FITNESS FOR A PARTICULAR PURPOSE, ARE HEREBY EXCLUDED, AND THE LIABILITY OF NETCOM,
IF ANY, FOR DAMAGES RELATING TO ANY ALLEGEDLY DEFECTIVE PRODUCT SHALL BE LIMITED
TO THE ACTUAL PRICE PAID BY THE PURCHASER FOR SUCH PRODUCT. IN NO EVENT WILL
NETCOM SYSTEMS BE LIABLE FOR COSTS OF PROCUREMENT OF SUBSTITUTE PRODUCTS OR
SERVICES, LOST PROFITS, OR ANY SPECIAL, DIRECT, INDIRECT, CONSEQUENTIAL OR INCIDENTAL
DAMAGES, HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, ARISING IN ANY WAY OUT OF
THE SALE AND/OR LICENSE OF PRODUCTS OR SERVICES TO RECIPIENT EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES AND NOTWITHSTANDING ANY FAILURE OF ESSENTIAL PURPOSE
OF ANY LIMITED REMEDY.
This portion of the user guide contains the following sections:
Section 1: Purpose
Section 2: Audience
Section 3: Manual Content
Section 4: Conventions Used in This Guide
Section 5: Related Manuals
Section 6: Online Help
Section 7: Technical Support
SmartApplications User Guide1
About this Guide
Purpose
Purpose
Audience
This user guide provides information on all procedures required to perform tests using
SmartApplications 2.30 application software. This includes details on software
installation, test setup, and test result interpretation.
This user guide is for users of SmartApplications 2.30 software. It is assumed that users of
this guide are familiar with Microsoft Windows and SmartBits equipment, and have an
intermediate knowledge level of data communications theory.
2SmartApplications User Guide
Manual Content
This guide contains the following chapters:
About this Guide
Manual Content
Chapter 1, “SmartApplications Overview.”
Chapter 2, “Basic Test Theory.”
Chapter 3, “Install and Connect.”
Chapter 4, “Basic Navigation and Tools.”
Chapter 5, “Set Up Tests.”
Chapter 6, “Run Tests.”
Chapter 7, “Test Scenario.”
Provides an overview of
SmartApplications.
Explains general SmartApplications
theory of operation and testing rules.
Lists system requirements and explains
how to connect your PC to a SmartBits
and DUT.
Describes the SmartApplications main
window and results windows, including
their menus and task buttons.
Explains procedures that are required to
set up any SmartApplications test.
Explains procedures to start, stop, and
run tests, and view reports.
Provides example test scenarios and
explains test results.
SmartApplications User Guide3
About this Guide
Conventions Used in This Guide
Conventions Used in This Guide
This guide uses the following conventions:
•Italics are used for document names and special terms.
•Menu options, field names, and tab names are bolded.
•Paths are shown with “greater than” symbols. For example, for Test > S et up Opt io ns
> Throughput. In this example, you would click on the Tes t menu option, then on
the Setup Options selection, and then on the Throughput tab.
•Notes are delineated by horizontal lines:
Note: Notes include related information, tips, and precautions about the topic preceding
them.
Related Manuals
Additional manuals related to this SmartApplications User Guide are listed below:
•SmartBits Getting Started
•SmartBits System Overview and Reference
•Using GPS with SmartBits
Online Help
SmartApplications provides online Help for all windows and tabs. You can access online
Help in two ways:
•Press the F1 key from the window about which you wish information.
•From the menu bar, select Help > Contents to view the entire contents of the Help
Chart Facility Help
SmartApplications includes a chart facility for graphically representing test results. It
contains a separate Help file titled First Impression. If you can access Help from a chartrelated window in either of the ways listed above, you will view chart-related Help only.
Once you return to the Results window, you can view SmartApplications Help.
file or Help > Search for Help On to search by a specific topic or word.
4SmartApplications User Guide
Technical Support
Technical support is available Monday through Friday between 07:00 and 18:00 Pacific
Standard Time.
To obtain technical support for any Netcom Systems product, please contact our Technical
Support Department using any of the following methods:
Phone: 800 886-8842 (available in the U.S. and Canada)
818 676-2589
Fax:818 880-9154
E-mail: support@netcomsystems.com
In addition, the latest versions of application Help files, application notes, and software
and firmware updates are available on our website at:
http://www.netcomsystems.com
Netcom Systems Headquarters
Netcom Systems, Inc.
26750 Agoura Road
Calabasas, CA 91302 USA
818 676-2300
About this Guide
Technical Support
SmartApplications User Guide5
About this Guide
Technical Support
6SmartApplications User Guide
1
SmartApplications Overview
What’s in this Chapter
This chapter contains the following sections:
Section 1: What is SmartApplications
Section 2: What are SmartCards, SmartModules and MiniModules
SmartApplications User Guide7
Chapter 1: SmartApplications Overview
What is SmartApplications
What is SmartApplications
SmartApplications is an easy-to-use suite of tests for LAN switches and routers
connecting to Ethernet, Token Ring, Frame Relay, and/or Token Ring networks. With
SmartApplications, you can test devices with up to 768 ports.
You can test how well a device performs with either packet-based or stream-based
network traffic that is simulated by SmartBits SmartCards. These tests are based on:
•RFC 1242, Benchmarking Terminology for Network Interconnection Devices, edited
by Scott Bradner (sob@harvard.edu)
•An informational document Benchmarking Methodology for Network Interconnect
Devices, edited by Scott Bradner
•RFC 2544, Benchmarking Methodology for Network Interconnect Devices, edited by
Scott Bradner and Jim McQuaid (mcquaidj@netscout.com)
RFC 1242 and RFC 2544 outline the following SmartApplications tests for network
devices:
•Throughput
•Latency
•Frame Loss Rate
•Back-to-Back
RFC 1242 contains a complete definition of the terms for which the tests are named. RFC
2544 describes how to measure and test performance, including specific formats in which
to report test results and maximum frame rates to use with specific frame sizes.
Both RFCs are included in an appendix to the SmartBits System Overview manual, and
RFC 1242 and the informational document are also available at the NDTL ftp site:
ndtl.harvard.edu
to view RFC 2544 or www.ietf.org.
8SmartApplications User Guide
Chapter 1: SmartApplications Overview
What are SmartCards, SmartModules and MiniModules
What are SmartCards, SmartModules and
MiniModules
SmartCards are custom-designed printed circuit boards (PCBs) that fit within a
SmartBits chassis to generate, capture and track network packet data. They fit into the
SMB-2000 and SMB-200 chassis.
SmartModules are made up of two PCBs within a specially designed tray which fits into
the SMB-6000 chassis. SmartModules provide a higher port density than SmartCards.
MiniModules consist of only one PCB, but have a higher port density than the
SmartCard. They are attached to a tray specially designed for the SMB-600.
The term “card” can refer to any SmartCard or any printed circuit board within a
SmartModule or MiniModule.
SmartApplications User Guide9
Chapter 1: SmartApplications Overview
What are SmartCards, SmartModules and MiniModules
10SmartApplications User Guide
2
Basic Test Theory
What’s in this Chapter
This chapter contains the following sections:
Section 1: Starting a Test
Section 2: General Testing Methodology
Section 3: Throughput Test Methodology
Section 4: Latency Test Methodology
Section 5: Packet Loss Rate Test Methodology
Section 6: Back-to-Back Test Methodology
SmartApplications User Guide11
Chapter 2: Basic Test Theory
Starting a Test
Starting a Test
When you have set up the comm port and test parameters and selected the ports to test,
you are ready to start running the test.
To start a test, you can either:
•Click the appropriate launch button on the main SmartApplications window
or
•Select Run from the menu bar.
You can also set all tests to run from the Preferences tab of the Test Setup window. The
next time you open SmartApplications they will begin running automatically.
Note: If you checked the Stop on Error box on the Test Setup window Preferences tab,
the test will stop running when it encounters error conditions such as “Latency packet not
received.” If you want the test to run regardless of errors, leave the box unchecked. If you
plan to run a long test, Netcom Systems recommends that you do NOT check this option.
Stopping a Test
To stop a test while it is running, click the Stop button on the Results window.
12SmartApplications User Guide
General Testing Methodology
Each SmartApplications test has these features in common:
•Measures device forwarding capabilities
•Requires a pair of SmartBits SmartCards to test frame forwarding: one is the
transmitter, the other is the receiver.
Before testing a device you must:
1Identify the number of the ports you want to test
2Ensure that each port to be tested has a unique MAC address
You can chose multiple packet sizes and test durations as specified in the RFC.
SmartApplications logs test results into individual standard ASCII files that you can view
or print with standard utilities.
Test Terminology
The term test refers to the test (such as Throughput or Latency) run for a particular frame
size.
Chapter 2: Basic Test Theory
General Testing Methodology
An iteration is when SmartApplications, while maintaining the frame size, varies the
transmission rate based on the last failed rate. The resolution and number of repetitions
determines the accuracy of results.
A trial is one set of iterations for a specific frame size and varying transmission rates.
Tests use trials to determine the optimal rate of traffic for that frame size. It is possible to
have multiple trials for the same frame size. The detailed results displays multiple trial
results. You can specify the number of trials in the Setup Test Configuration window.
The following diagram illustrates these terms:
SmartApplications User Guide13
Chapter 2: Basic Test Theory
General Testing Methodology
Address Learning for SmartCards
SmartApplications assumes that the device under test (DUT) has no knowledge of any
forwarding or routing required. Device forwarding tables are initialized by learning
packets. You can also send learning packets in between every test trial in order to refresh a
device forwarding table before an address is eliminated by aging.
This table lists the contents of learning packets:
Table 2-1.Learning Packet contents
AreaSizeHow Used
Destination6MAC address
Source6MAC address
ProtocolvariableMatching user selection of IP, UDP, IPX, or 32 bytes of All
zeros, or All 0xFF's
Test type12For Packet Loss and Back-to-Back: 6 bytes of the Netcom
stamp in an ASCII string, and 6 bytes of the destination
address.
For Throughput: a 6 byte incrementing counter with 6 bytes of
destination MAC address.
For Latency: a 6 byte decrementing counter with 6 bytes of
destination MAC address.
Rest of packetnQuantity varies depending on the selected protocol type's fill
pattern.
CRC4
14SmartApplications User Guide
Throughput Test Methodology
The Throughput test tests the fastest rate at which a device can forward frames without
error. If a single frame is dropped the test fails and the test is repeated at a lower
throughput rate.
The throughput test satisfies the terminology criteria of RFC 1242 and the test
methodology specified in RFC 2544. From RFC 1242, throughput is “the maximum rate
at which none of the offered frames are dropped by the device.” From RFC 2544, the
methodology to measure throughput is to “Send a specific number of frames at a specific
rate through the DUT and then count the frames that are transmitted by the DUT.”
SmartApplications measures throughput by first sending a series of frames with a
particular source and destination MAC address to the device under test. These frames are
sent from a particular SmartBits port intended to be received by another SmartBits port.
Gap is determined by the initial rate in the test setup. If the initial rate is set to 100%,
frames are sent initially with the gap set to the minimum legal size for the topology and
speed of the transmitting SmartCard. The number of frames sent is determined by the
length of time the test is to run. Frames are validated by counting only frames generated
by the sending port, not any packets sent by the switch. Keep-alive and routing update
frames are not counted as received frames.
Chapter 2: Basic Test Theory
Throughput Test Methodology
If the initial rate is 100%, the first throughput test trial frame rate for a given frame length
is the maximum rate at for the topology and speed of the transmitting SmartCard. If all
frames from the transmitting SmartBits port are received by the receiving port, no further
trials are attempted and the maximum frame rate is recorded as the throughput.
If the first trial fails (if even a single frame is lost), the second trial frame rate drops to
20% lower than the rate at which it failed. The third and each subsequent trial uses a
binary search to determine a rate that is halfway between the last failed rate and the last
successful rate. The test will not finish until the frame loss percent is less than or equal to
the resolution value in the test setup.
Results of the test are automatically logged to a file showing maximum data rate by port
pair and aggregate frame data rate. Percentage achieved of maximum theoretical frame
data rate for each selected frame size is also logged.
This test is performed for a user-specified number of times and the results averaged.
SmartApplications User Guide15
Chapter 2: Basic Test Theory
Latency Test Methodology
Latency Test Methodology
The SmartApplications Latency test measures latency as defined in RFC 1242.
How SmartApplications Calculates Latency
The sending SmartCard sends a burst of frames at a user-specified frame size through the
DUT at a user-specified throughput rate. In the middle of the burst stream it inserts one
frame with an identifying trigger (tag). The time when the trigger frame is fully
transmitted is the Transmit Timestamp. The time the receiving SmartCard recognizes the
trigger frame is the Receive Timestamp. SmartApps calculates Latency as:
(Receive Timestamp) minus (Transmit Timestamp) = Latency
Specifically, SmartApps first measures the cut-through rate and then calculates the store
and forward rate with this calculation:
(Cut Through Rate) minus (Frame Bit Time) = Store and Forward Rate
The Latency test is performed at a user-specified frame rate.
Cut Through (Bit-forwarding) Latency Calculation
For Cut Through device measurements, SmartApps calculates Latency as FIFO (first infirst out), meaning it calculates the difference between the time that the end of the first bit
of a transmitted frame reaches the DUT's input port (Transmit Timestamp) and the time
that the end of the first bit of the same frame leaves the DUT's output port (Receive
Timestamp).
Store and Forward Device Latency Calculation
For Store and Forward device measurements, SmartApps calculates Latency as LIFO (last
in-first out), meaning it calculates the difference between the time that the end of the last
bit of a transmitted frame reaches the DUT's input port (Transmit Timestamp) and the
time that the end of the first bit of the same frame leaves the DUT's output port (Receive
Timestamp).
Note:
1) It is recommended that you run the Throughput test before you run the Latency test in
order to obtain the optimum throughput rate of the DUT. Then use the throughput rate
obtained from the Throughput test for the Max Rate in the Latency test. Using a proven
optimum rate prevents the tagged frame from being lost due to performance rate issues.
2) SmartApplications always measures Latency and yields results for both store and
forward devices and bit-forwarding (cut-through) devices regardless of the DUT. When
viewing results, use the result that is appropriate for your type of device.
16SmartApplications User Guide
Packet Loss Rate Test Methodology
This test measures the percentage of frames lost by the DUT that should have been
forwarded.
The Frame Loss Rate test satisfies the terminology criteria of RFC 1242 and the test
methodology specified in RFC 2544. From RFC 1242, Frame Loss Rate is the
“Percentage of frames that should have been forwarded by a network device under steady
state (constant) load that were not forwarded due to a lack of resources”. The test allows
for varying frame sizes from 64 bytes to 1518 bytes for Ethernet ports, and up to 8188 for
Token Ring ports.
From RFC 2544, the methodology to measure frame loss is to “Send a specific number of
frames at a specific rate through the DUT to be tested and count the frames that are
transmitted by the DUT.”
Frames are validated by counting only frames generated by the sending port, not any
packets sent by the switch. Keep-alive and routing update frames are not counted as
received frames.
The Frame Loss Rate test operates in essentially the same manner as the Throughput test.
First a packet burst is performed at the maximum possible rate for a user-specified period
of time. After all packets are sent, the receiving port is queried to determine how many
packets were received. The number of packets not received is determined and the
percentage of loss is calculated.
Chapter 2: Basic Test Theory
Packet Loss Rate Test Methodology
Note: For Many to 1/1 to Many test configurations between ATM and Ethernet or ATM
and Frame Relay: if the DUT bundles multiple streams into one virtual circuit, SmartApps
counters reflect the number of packets received on a per virtual circuit basis (not per
stream).
This test is performed a user-specified number of times and the results averaged.
SmartApplications User Guide17
Chapter 2: Basic Test Theory
Back-to-Back Test Methodology
Back-to-Back Test Methodology
Back-to-Back tests the buffering capability of the device under test.
The Back-to-Back test satisfies the terminology criteria of RFC 1242 and the test
methodology specified in RFC 2544. From RFC 1242, the Back-to-Back test is performed
by “fixed length frames presented at a rate such that there is the minimum legal separation
(maximum rate) for a given medium between frames over a sort to medium period of time,
starting from an idle state.” The test allows for varying frame sizes from 64 bytes to 1518
bytes for Ethernet ports, and 8188 for Token Ring ports.
From RFC 2544, the methodology to measure back-to-back frames is to “Send a burst of
frames with minimum inter-frame gaps to the DUT and count the number of frames
forwarded by the DUT.” Frames are validated by counting only frames generated by the
sending port, not any packets sent by the switch. Keep-alive and routing update frames are
not counted as received frames.
The Back-to-Back test operates in essentially the same manner as the Throughput test.
First a packet burst is performed for a user specified period. If all packets are received at
the receiving port, the test is successful and testing is stopped. If even one packet is lost,
the number of packets sent in the burst is halved and retried. If all packets are successfully
received at this point, the packet burst is then chosen half way between the successful and
unsuccessful trials, and retried. This pattern repeats until the actual number of packets that
can be forwarded in a burst is determined.
This test is performed a user-specified number of times and the results are averaged.
18SmartApplications User Guide
3
Install and Connect
What’s in this Chapter
This chapter contains the following sections:
Section 1: Installation
Section 2: Open SmartApplications
SmartApplications User Guide19
Chapter 3: Install and Connect
Installation
Installation
It is assumed that you are familiar with Microsoft Windows and basic SmartBits
operation.
System Requirements
To run SmartApplications 2.30, you must have the following:
Hardware
•SMB-2000/200 or SMB-6000/600
•At least two SmartCards
•IBM PCÔ or 100% compatible
•Pentium 200 or above
•VGA video or better is recommended
•16 MB of RAM for Windows 95 / 98
•32 MB of RAM for Windows NT
•10 MB free hard disk space
Software
•Windows 95Ô,Windows98Ô or Windows NTÔ 4.0
•SmartBits Firmware 10.10 or higher
•SmartApplications software
Installing from a CD
Please refer to installation instructions on the SmartApplications CD case.
Installing from Windows 95 or Windows NT
SmartApplications is provided on one CD-ROM disk. To install SmartApplications,
perform the following steps:
1Insert the SmartApplications installation CD in the CD-ROM drive.
The Install Shield window is displayed.
2Select Install SmartApplications [Ver#].
3Follow the instructions until all Netcom software is installed.
20SmartApplications User Guide
Chapter 3: Install and Connect
Installation
If your PC does not support the automatic install process, you can use the following
alternative procedure.
1From the Start Menu, select Run.
2When the Run dialog box appears, type "
<cd-rom drive>:\SETUP"intheCommand
Line, then click the OK button.
3Follow instructions as the installation program installs the software.
SmartApplications User Guide21
Chapter 3: Install and Connect
Open SmartApplications
Open SmartApplications
From the Start menu, select Programs > SmartApplications. The SmartApplications
main window is displayed.
Close SmartApplications
To close SmartApplications, click the Close button at the top right corner of the screen.
or
Select File > Exit from the menu bar.
22SmartApplications User Guide
4
Basic Navigation and Tools
What’s in this Chapter
This chapter contains the following sections:
Section 1: The Main Window
Section 2: Charting Test Results
SmartApplications User Guide23
Chapter 4: Basic Navigation and Tools
The Main Window
The Main Window
The SmartApplications main window is a dynamic window that changes according to the
selections you make, such as selecting 1 to Many or 1 to 1 testing.
Parts of the Main Window
Menu Bar
To ol B a r
Test Launch
buttons
Por t Select ion
area
Card Attributes
area
Connection
status
24SmartApplications User Guide
Loading...
+ 126 hidden pages
You need points to download manuals.
1 point = 1 manual.
You can buy points or you can get point for every manual you upload.